CAREER: Exploring Non-Euclidean Representation Learning for Expressive and Explainable Graph-based Foundation Models

This project addresses a critical limitation in current artificial intelligence systems: their inability to accurately understand and represent the complex network-like relationships that exist in the real world. Existing state-of-the-art models lack flexibility when modeling such data in areas like biological networks, drug discovery, or scientific literature retrieval systems, fundamentally due to the disparity between the complex structure of the data, and the Euclidean geometry properties of AI internal representations. By developing a new class of AI that uses more flexible mathematical shapes, this project will enable AI to better reflect the way entities like molecules or cells interact in their space. These advancements can significantly improve human health and national welfare by accelerating the discovery of new medicines, enhancing our understanding of age-related diseases, and creating more reliable tools for scientific discovery. Furthermore, the project improves the safety of AI by making these complex systems more explainable, allowing experts to understand and trust the reasoning behind AI prediction in high-stakes environments like healthcare. The research will involve participants at all levels from high-school students to graduate students through outreach programs and facilitate development of new university courses in modern multimodal foundation models and trustworthy AI. The project aims to build the next-generation AI foundation model that can capture complex topological relationships in its embedding spaces. The methodology focuses on developing the first fully flexible and expressive foundation model that utilizes adaptive curvature in a non-Euclidean embedding space. The goal is achieved via a novel geometric approach including a non-Euclidean graph Transformer architecture and a new learnable curvature estimation method. The model is further extended via multimodal fusion that integrates the graph Transformer with a non-Euclidean large language model. The resulting framework will achieve simultaneous geometric modeling of graph relational information and scale-free token distributions in natural language. Furthermore, the framework will be enhanced by intrinsic explainability using attention and information bottleneck. Finally, the complex geometric representations will be translated into human-readable narratives. The project utilizes a cross-domain pre-training strategy to ensure the resulting model is universally applicable and can be applied to diverse fields including spatial transcriptomics, molecular property prediction, and scientific literature analysis. This work contributes to the field of machine learning by providing a theoretically grounded approach for capturing non-Euclidean characteristics in large-scale data prevalent in biological and physical sciences, facilitating inter-disciplinary education and training. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria. NSF Award ID: 2540656 | Program: 01002627DB NSF RESEARCH & RELATED ACTIVIT,01003031DB NSF RESEARCH & RELATED ACTIVIT,01002930DB NSF RESEARCH & RELATED ACTIVIT | Principal Investigator: Rex Ying | Institution: Yale University, NEW HAVEN, CT | Award Amount: $596,523 View on NSF Award Search: https://www.nsf.gov/awardsearch/show-award/?AWD_ID=2540656 View on Research.gov: https://www.research.gov/awardapi-service/v1/awards/2540656.html