Intelligent Agents in the Classroom: Supporting Machine Learning Understanding Through Collaborative Design

This project aims to serve the national interest by advancing undergraduate students’ artificial intelligence (AI) literacy through innovative, experiential learning that strengthens career readiness, civic preparedness, and broad STEM engagement. This Level 3 Engaged Student Learning project intends to increase students’ ability to understand, critique, and thoughtfully engage with AI as part of everyday life and future professional practice. The project addresses the national importance of ensuring that all undergraduates develop the conceptual foundations needed to navigate AI-enabled technologies, participate in public discourse, and make informed decisions. To meet this need, the project plans to create and test an adaptable three-week, hands-on learning unit that uses a physical model for machine learning concepts, supported by an intelligent conversational agent. This approach incorporates principles from the Science of Human Learning and makes abstract AI ideas tangible and accessible. By integrating collaborative design, tangible making, and AI-supported learning, the project aims to produce positive outcomes such as improved conceptual understanding, strengthened interdisciplinary problem solving, and increased student confidence in reasoning about AI systems. The project’s significance lies in its potential to model how undergraduate education can meaningfully prepare a wide range of learners to participate in an AI-shaped society while also offering a scalable, evidence-based instructional model for diverse institutions. The project’s technical goals and scope include designing, implementing, and studying a 3-week AI-enabled STEM learning unit grounded in Learning by Design principles. The unit centers on a tangible representation of matrix factorization, the mathematical foundation of machine learning, using a shaft loom as a boundary object through which students explore abstraction, pattern decomposition, compression, generalization, and overfitting. The research plan will investigate how loom-based physical constraints provide scaffolding to understand machine learning representations; which curriculum design features enable students to demonstrate mastery of decomposition and abstraction; how collaborative design processes between curriculum developers and instructors support disciplinary adaptation; and how an AI conversational agent can adjust its scaffolding to accommodate students with varied backgrounds in mathematics, programming, arts, and design. The project plans to use mixed-methods research, including learning analytics, performance assessments, and qualitative analysis of student interactions, to evaluate learning trajectories and compare implementations across multiple instructional contexts. Results will be disseminated through open-access curriculum materials, faculty workshops, and peer-reviewed publications. The NSF IUSE: EDU Program supports research and development projects to improve the effectiveness of STEM education for all students. Through the Engaged Student Learning track, the program supports the creation, exploration, and implementation of promising practices and tools. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria. NSF Award ID: 2540049 | Program: 04002627DB NSF STEM Education | Principal Investigator: Carolyn Rose | Institution: Carnegie Mellon University, PITTSBURGH, PA | Award Amount: $923,193 View on NSF Award Search: https://www.nsf.gov/awardsearch/show-award/?AWD_ID=2540049 View on Research.gov: https://www.research.gov/awardapi-service/v1/awards/2540049.html