CAREER: Beyond Multi-Index Models: Statistical and Algorithmic Foundations for Feature Learning in Compositional Architectures

Across science and engineering, from biology, econometrics to aerodynamics, data are often high-dimensional and complex, yet the outcomes of interest are frequently governed by only a few critical factors interacting in structured ways. Despite major advances, a significant gap remains between interpretable statistical methods and modern high-performing predictive systems. Classical statistical approaches, such as multi-index models, seek to capture these mechanisms through compositions of linear and nonlinear transformations, but many existing methods rely on carefully tuned parameters that can reduce stability and reproducibility. Meanwhile, modern machine learning emphasizes predictive performance through compositional architectures such as deep neural networks, yet there is limited theoretical understanding of when these models recover meaningful underlying structure rather than fit patterns through trial and error. This project addresses that gap by developing compositional learning models, together with new theory and algorithms, that combine rigorous statistical foundations with the ability to reliably uncover predictive structure in complex data. The research efforts are integrated with educational activities including mentoring undergraduate and graduate students, developing new courses in statistics and machine learning at Northwestern University, and disseminating methods through open-source software and interdisciplinary collaborations. To achieve these goals, this project develops a unified framework for learning compositional models and understanding when such models discover true low-dimensional structure in data. Statistical advances include multi-index modeling by jointly learning the linear and nonlinear transformations through direct optimization of a compositional objective. In addition, defying conventional expectation, this approach recovers the underlying low-dimensional structure without explicit penalization on the linear components and thus reduces the need for hyperparameter tuning. Machine learning and artificial intelligence (AI) advances include new statistical principles clarifying when compositional architectures trained purely via empirical risk minimization recover low-dimensional representations that capture the underlying mechanisms driving complex data, without added regularization. These foundations promise not only practical tools for scientists seeking interpretable models with less tuning, but also principles to guide the design of more structured, robust, and data-efficient AI systems, which are increasingly reliant on compositional architectures. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria. NSF Award ID: 2540678 | Program: 01002930DB NSF RESEARCH & RELATED ACTIVIT,01003031DB NSF RESEARCH & RELATED ACTIVIT,01002627DB NSF RESEARCH & RELATED ACTIVIT | Principal Investigator: Feng Ruan | Institution: Northwestern University at Chicago, EVANSTON, IL | Award Amount: $244,168 View on NSF Award Search: https://www.nsf.gov/awardsearch/show-award/?AWD_ID=2540678 View on Research.gov: https://www.research.gov/awardapi-service/v1/awards/2540678.html