CAREER: Harnessing the Power of Off-Dynamics Reinforcement Learning: Foundations, Algorithms, and Applications

Many important decisions in everyday life, such as choosing effective medical treatments, managing public health responses, or controlling autonomous systems, must be made step by step while learning from limited and imperfect information. Current artificial intelligence methods often require extensive trial-and-error interactions with the real world to learn effective strategies, which is impractical or unsafe in high-stakes settings where mistakes are costly or unethical. This project addresses this challenge by developing new approaches that allow intelligent systems to learn from simulated or indirect environments and reliably transfer that knowledge to real-world situations, even when conditions differ. By enabling safer and more data-efficient decision-making, the project has the potential to improve technologies in healthcare, robotics, and other critical domains, ultimately benefiting public health, economic productivity, and societal well-being. The project will also contribute to education and workforce development by training students at multiple levels, creating accessible learning materials, and conducting outreach activities to broaden participation in artificial intelligence. All resulting software, data resources, and educational materials will be made openly available to maximize their impact. This project develops a comprehensive theoretical and algorithmic framework for off-dynamics reinforcement learning, which studies how to train decision-making agents in a source domain, such as a simulator, and effectively deploy them in a target domain with different and potentially unknown transition dynamics. The research addresses fundamental challenges arising from distributional shifts between training and deployment environments. The work is organized into three main research activities: (1) developing distributionally robust learning methods that ensure reliable performance via minimax optimization over an uncertainty set of transitions when the target domain is unknown, supported by finite-sample performance guarantees; (2) designing algorithms that leverage partial access to target-domain data through data augmentation and cross-domain learning to improve transfer efficiency; and (3) establishing new frameworks for learning under limited interaction and high policy-switching costs, focusing on stability and efficiency in real-world deployment. The proposed methods will be analyzed theoretically to characterize statistical limits and performance guarantees and will be validated empirically on standard reinforcement learning benchmarks and real-world healthcare datasets. All developed algorithms will be released as open-source implementations to support reproducibility and broad adoption. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria. NSF Award ID: 2541960 | Program: 01002627DB NSF RESEARCH & RELATED ACTIVIT,01003031DB NSF RESEARCH & RELATED ACTIVIT,01002930DB NSF RESEARCH & RELATED ACTIVIT | Principal Investigator: Pan Xu | Institution: Duke University, DURHAM, NC | Award Amount: $420,000 View on NSF Award Search: https://www.nsf.gov/awardsearch/show-award/?AWD_ID=2541960 View on Research.gov: https://www.research.gov/awardapi-service/v1/awards/2541960.html