Reliable Methods for Estimation, Prediction and Causal Inference with Multiple AI-Generated Synthetic Datasets

Artificial intelligence (AI) systems, including modern machine learning models and large language models, are now routinely used to generate predicted labels and synthetic data across science, medicine, and policy. Researchers increasingly rely on AI-generated datasets to replace expensive or scarce human-labeled data, but the quality of such outputs is often unknown and can vary widely across AI systems. Treating AI predictions as ground truth can lead to incorrect scientific conclusions, misleading policy recommendations, and overconfident uncertainty quantification—risks that grow as AI is deployed in higher-stakes settings, such as clinical research and public health. This project develops the statistical foundations needed to use AI-generated data in scientific analysis. By turning AI-generated data from a potential liability into a rigorously calibrated scientific resource, the project advances national priorities in science, health, and the responsible use of AI. The project also supports U.S. workforce development through training of graduate and undergraduate students, integration of research outcomes into university curricula, and the public release of open-source software that makes the methodology broadly accessible to scientists, agencies, and industry. This project develops a unified statistical theory for the safe and adaptive integration of multiple, heterogeneous, and potentially low-quality AI-generated synthetic datasets into estimation, prediction, and causal inference. The framework treats synthetic data as auxiliary information whose contribution must be calibrated to ensure no loss relative to labeled-data baselines, while automatically exploiting informative sources when available. The research is organized around three connected aims. Aim 1 advances semiparametric estimation and inference by constructing unbiased, efficiency-improving procedures that optimally combine multiple synthetic datasets. Aim 2 targets prediction accuracy rather than estimation efficiency, developing non-asymptotic excess-risk guarantees that ensure prediction performance is never worse than labeled-only baselines and can be substantially better when informative synthetic labels exist. Aim 3 introduces a new framework for causal inference in multi-site randomized controlled trials augmented with synthetic data, addressing site heterogeneity and privacy constraints by integrating causal estimands, semiparametric efficiency bounds, and privacy-preserving computation. Together, the three aims provide rigorous statistical foundations for AI-augmented evidence generation, and their methodological contributions will be disseminated through peer-reviewed publications, open-source software, conference presentations, and graduate-level course materials. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria. NSF Award ID: 2610561 | Program: 01002627DB NSF RESEARCH & RELATED ACTIVIT | Principal Investigator: Jiwei Zhao | Institution: University of Wisconsin-Madison, MADISON, WI | Award Amount: $250,000 View on NSF Award Search: https://www.nsf.gov/awardsearch/show-award/?AWD_ID=2610561 View on Research.gov: https://www.research.gov/awardapi-service/v1/awards/2610561.html