Stable formulations and numerical discretizations of loss functions for generative AI with applications to model reduction

Artificial intelligence is increasingly used to model complex scientific and engineering systems. Maintaining leadership in this area over the long term depends not only on larger models and more data but also on mathematical methods to enhance training and to make training more stable, reproducible, and mathematically well-founded. This project will develop mathematical principles for designing training objectives and methods for generative artificial intelligence that remain stable under realistic training conditions, such as finite data, rather than producing misleading updates or fragile models. This project directly advances artificial intelligence as an area of federal strategic interest by strengthening the foundations needed for reliable artificial intelligence models in scientific and engineering applications. More broadly, this project will strengthen the scientific and industrial artificial intelligence ecosystem in the United States by developing reliable methods for discovery, design, and decision-making in complex systems. The project will also support education and workforce development through graduate student training, integration of project ideas into courses, open source software, and public benchmark problems. The project will develop stable formulations and numerical discretizations of loss functions for generative artificial intelligence models of time-dependent stochastic processes. These models often use training objectives involving time and space derivatives, but current practice commonly estimates such objectives from samples in a black box manner, which can introduce systematic errors, poor conditioning, and unstable training. The project will show that loss functions for certain generative models can be interpreted as variational formulations of partial differential equations. This connection will make it possible to transfer concepts such as stability, coercivity, structure preservation, and consistent discretization into the design of empirical loss functions for data-driven generative modeling. The work will establish rigorous correspondences between continuous loss functions and partial differential equation formulations, derive well-conditioned discrete loss functions that remain stable when only data samples are available, and develop discretization strategies that also provide algorithmic benefits such as parallel training across time. The resulting methods will be applied to reduced and surrogate modeling of stochastic and chaotic systems, to demonstrate more stable, reliable, and efficient generative reduced models. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria. NSF Award ID: 2608887 | Program: 01002627DB NSF RESEARCH & RELATED ACTIVIT | Principal Investigator: Benjamin Peherstorfer | Institution: New York University, NEW YORK, NY | Award Amount: $397,458 View on NSF Award Search: https://www.nsf.gov/awardsearch/show-award/?AWD_ID=2608887 View on Research.gov: https://www.research.gov/awardapi-service/v1/awards/2608887.html