Understanding the Role of Metacognitive Sensitivity in Human-AI Joint Decision-Making

Human beings increasingly make important decisions with artificial intelligence (AI) systems in domains such as medicine, education, science, business, and national security. While AI systems can provide valuable assistance, currently human–AI teams do not always make better decisions than human or AI systems working alone. This project investigates when AI improves human decision-making. The research examines how people decide whether or not to defer to an AI system that provides both recommendations and estimates of confidence. By identifying the conditions under which human-AI teams succeed, the project has the potential to provide guidance for designing AI systems that promote effective learning and better decision making. The project tests how confidence, accuracy, and communication shape decision-making in human–AI teams. Through a series of behavioral experiments, human participants and multimodal AI agents make decisions, report their confidence, exchange information through language, and then have the opportunity to revise their decisions and confidence ratings over time. The research focuses on two key metacognitive properties: metacognitive sensitivity, which is how well confidence tracks whether a decision is correct, and metacognitive calibration, which is whether confidence is generally too high, too low, or appropriately matched to performance. The research systematically varies AI performance, AI metacognitive sensitivity, and AI metacognitive calibration in both static and changing environments. These experiments aim to show how people learn to interact with AI partners effectively, and how this evolves as AI systems are updated, retained, or repurposed. Computational models of decision-making, confidence, and learning are used to identify the mechanisms that support successful human-AI collaboration. By uncovering how people learn to evaluate and adapt to intelligent systems, the project advances the science of learning and provides foundational knowledge for designing AI technologies that not only perform well but also help people learn, adapt, and make better decisions over time. Educational activities associated with the project engage undergraduate and graduate students in interdisciplinary research at the intersection of cognitive science, learning science, and artificial intelligence. Open educational resources and interactive demonstrations aim to promote AI literacy and responsible AI use among the broader public. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria. NSF Award ID: 2617585 | Program: 01002627DB NSF RESEARCH & RELATED ACTIVIT | Principal Investigator: Brian Odegaard | Institution: University of Florida, GAINESVILLE, FL | Award Amount: $890,113 View on NSF Award Search: https://www.nsf.gov/awardsearch/show-award/?AWD_ID=2617585 View on Research.gov: https://www.research.gov/awardapi-service/v1/awards/2617585.html