CAREER: Assured Reinforcement Learning from Human Feedback for Cyber-Physical Systems

This CAREER project will develop new methods that allow cyber-physical systems, such as robotaxis, service robots, and autonomous drones, to adapt to individual human preferences using simple feedback. Today, many such systems rely on fixed rules designed in advance, which makes it difficult for them to respond to differences in how people prefer to work or interact with them. This project will enable these systems to learn from intuitive input that non-expert users can provide, such as choosing between options or ranking outcomes, while maintaining safety, privacy, and security during operation. The project will develop new theory and algorithms, release open-source tools, and validate the resulting methods on real-world robotic platforms. It will also advance education and workforce development through new course materials, immersive learning platforms, student mentoring, and outreach activities for undergraduate and K-12 students, in collaboration with the University of Florida Transportation Institute, the Florida Institute for National Security, and the Florida Institute for Cybersecurity Research. This project will establish a comprehensive framework for assured reinforcement learning from human feedback for cyber-physical systems, enabling systems to learn personalized control policies directly from simple human feedback while providing guarantees for safe, secure, and privacy-preserving operation. The research is organized into three integrated thrusts. Thrust 1 will develop logic-aware methods that translate simple human feedback into logic-based reward models to update system control policies, addressing noisy or limited human feedback through uncertainty quantification, robust estimation, and active learning. Thrust 2 will develop methods for deploying these systems in human environments with temporal constraints, including both predefined constraints and those inferred from human interaction, while supporting bounded violation during exploration, risk awareness for worst-case outcomes, and robustness to distribution shifts. Thrust 3 will address privacy and security concerns associated with integrating human feedback into cyber-physical systems by protecting human data, defending against data poisoning attacks, and investigating federated learning architectures for large-scale systems. All developed methods will be evaluated through real-world case studies involving autonomy with human interaction. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria. NSF Award ID: 2541603 | Program: 01002627DB NSF RESEARCH & RELATED ACTIVIT | Principal Investigator: Yu Wang | Institution: University of Florida, GAINESVILLE, FL | Award Amount: $659,317 View on NSF Award Search: https://www.nsf.gov/awardsearch/show-award/?AWD_ID=2541603 View on Research.gov: https://www.research.gov/awardapi-service/v1/awards/2541603.html