CAREER: Supporting Non-AI Experts in Living and Working with Imperfect Artificial Intelligence

Artificial intelligence (AI) systems built on machine learning have inherent limitations due to training and data quality, and may appear surprisingly unintelligent when their behavior does not match what people expect from an intelligent system. Despite these limitations, imperfect AI can still be very helpful and has been widely adopted, raising concerns about being left behind as these technologies advance. This project seeks to support non-AI experts in living and working effectively with imperfect artificial intelligence tools by studying the workarounds people generate when encountering imperfect AI, what affects their ability to adapt these AI tools to their needs, and what knowledge and skills should be prioritized when equipping non-AI experts to work with such tools. The insights gained can inform the design of human-AI interfaces and the development of training programs, enabling the general public to better utilize AI tools. The project will also identify essential competencies for AI literacy, helping equip the future workforce with the ability and confidence to explore, question, and critically assess new AI tools they encounter. To support non-AI experts in living and working effectively with imperfect AI, the project includes four research tasks that combine mixed-methods studies and controlled experiments. The first task is to develop and validate a general framework of AI-related workarounds by collecting a diverse sample of non-expert users adapting to imperfect AI tools. The second task focuses on studying factors that influence workaround generation. It involves exploring how users’ mental model, sensemaking, and trust calibration relate to workaround generation. It also investigates the influence of task effort and time constraints. The third task focuses on designing and testing interventions to support effective workarounds. It compares three types of training, namely machine learning knowledge, failure cases, and workaround strategies. It also examines whether just-in-time hints can inspire and encourage effective workarounds without requiring prior training. The fourth task explores whether and how skills and insights gained in the context of one AI tool may transfer across different tasks and tools. Together, these tasks will advance knowledge in human-AI interaction and AI literacy. The project establishes workarounds as an important lens for understanding human adaptability and resiliency in navigating imperfect AI. It also contributes to defining essential competencies for AI literacy and has implications for human-AI interface design that supports resilient human-AI partnerships. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria. NSF Award ID: 2543108 | Program: 01002627DB NSF RESEARCH & RELATED ACTIVIT,01002930DB NSF RESEARCH & RELATED ACTIVIT,01003031DB NSF RESEARCH & RELATED ACTIVIT | Principal Investigator: Xiaomei Wang | Institution: University of Louisville Research Foundation Inc, LOUISVILLE, KY | Award Amount: $331,717 View on NSF Award Search: https://www.nsf.gov/awardsearch/show-award/?AWD_ID=2543108 View on Research.gov: https://www.research.gov/awardapi-service/v1/awards/2543108.html