Collaborative Research: Reinforcement Learning with High-Probability Safety constraints: Theory, and Applications

This NSF project aims to make reinforcement learning (RL) safe enough for real-world cyber-physical systems, such as unmanned aerial vehicles, mobile robots, and networked infrastructures, where rare failures can cause disproportionate harm. Today, many learning-enabled systems optimize average performance, yet safety-critical deployments require strong guarantees that unsafe events remain exceedingly unlikely, even when conditions change unexpectedly (e.g., sudden obstacles, sensor dropouts, or model mismatch between simulation and reality). The project will bring transformative change by developing a new foundation for “high-confidence autonomy,” where learning algorithms explicitly respect stringent safety constraints while still improving performance. This will be achieved by designing learning methods that quantify uncertainty, constrain exploration, and adapt policies cautiously when the environment shifts. The intellectual merit of the project includes new theory and algorithms that enable safe decision-making with explicit probabilistic guarantees under model uncertainty, limited data, limited observation, and changing operating conditions. The broader impacts of the project include advances in reliable autonomy relevant to national health, prosperity, welfare, and defense; open-source software and benchmark scenarios that lower barriers to safe autonomy research; and training of students through curriculum modules and research experiences that broaden participation in STEM education. Today’s RL methods primarily optimize expected performance and typically provide only average safety, which is insufficient for safety-critical autonomy. This project will develop a new foundation for high-probability safe RL, where safety requirements are enforced as explicit probabilistic guarantees (e.g., safety holds with a high probability), not just in expectation. The research will deliver theory and algorithms that (i) learn safe policies efficiently in simulation, (ii) remain safe under simulator-to-real mismatches, (iii) explore and improve online through “safe exploration” that limits violations during learning, and (iv) maintain safety when state information is noisy or partially missing (e.g., GPS-denied or cluttered environments). The team will validate the resulting methods in high-fidelity simulation and on real UAV platforms in representative scenarios, including wildfire monitoring and energy-efficient soaring missions. Outcomes will advance trustworthy autonomy for cyber-physical systems and provide broadly reusable tools for safety-critical RL in robotics and networked systems. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria. NSF Award ID: 2534262 | Program: 01002627DB NSF RESEARCH & RELATED ACTIVIT | Principal Investigator: Arnob Ghosh | Institution: New Jersey Institute of Technology, NEWARK, NJ | Award Amount: $224,996 View on NSF Award Search: https://www.nsf.gov/awardsearch/show-award/?AWD_ID=2534262 View on Research.gov: https://www.research.gov/awardapi-service/v1/awards/2534262.html