Cognitive and Neural Representations of Visual Scenes

This project investigates our mental representations of surrounding space, objects and layout, that help us make decisions about information in our visual world that in turn drive our behaviors. When we are “at the beach” or “in the kitchen” we do not see the entire scene all at once, but we can use our knowledge about the context to make predictions about what information is missing and use that information to decide how to move our eyes, head, and body to navigate in the space. Cognitive and neuroimaging experiments probe how our minds and brains use what they expect to see, rather than solely what is in the visual scene, to adjudicate among competing theories of how observers anticipate and remember information in scenes and how attention, expectation, and visual perception interact. The investigators 1) contrast two distinct conceptualizations of scene representation, a spatio-centric view, which relies on generative scene processing, and a visuo-centric view, which relies on visible scene content alone; 2) construct stimuli based on their visual characteristics as well as on predictions they evoke about expected surroundings; and 3) develop and extend a promising new multivoxel pattern analysis (MVPA) method to “decode” extended space and thus interrogate the neural correlates of extensions of spatial expanse and content. The work offers a new reference point for comparing human scene understanding with that of artificial intelligence systems, including image and language models such as ChatGPT. It also includes the preservation and open sharing of photograph and drawing sets from pioneering scene-perception studies dating to the 1960s and 1970s. These materials that are valuable to scientists, classroom teachers, and science museums but are at risk of being lost without digitizing. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria. NSF Award ID: 2545451 | Program: 01002627DB NSF RESEARCH & RELATED ACTIVIT | Principal Investigator: Helene Intraub | Institution: University of Delaware, NEWARK, DE | Award Amount: $693,716 View on NSF Award Search: https://www.nsf.gov/awardsearch/show-award/?AWD_ID=2545451 View on Research.gov: https://www.research.gov/awardapi-service/v1/awards/2545451.html