CAREER: Aligning Computational Efficiency and Human Perception in Visual Generative Models

Recent advances in generative artificial intelligence (GenAI) have transformed the way digital images and videos are created, enabling the automated generation of highly realistic content. However, these powerful technologies require massive amounts of electricity and computer memory, leading to high operational costs and significant infrastructure requirements. While these systems consume vast resources, the human eye and brain can only process a limited amount of visual information at any given time. This project addresses the gap between the high computational cost and the actual visual quality experienced by a human viewer. By aligning the efficiency of computer systems with the limits of human vision, this work aims to create cost-effective and faster digital content generation tools. This research supports the national interest by promoting economic prosperity through reduced industrial expenses and by accelerating the development of larger-scale, more efficient generative artificial intelligence infrastructure. This project establishes a research framework to measure, model, and optimize the relationship between computational expense and human visual perception for emerging GenAI models. The research methodology involves the development of psychophysical studies and large-scale datasets to quantify how specific hardware costs correlate with perceived visual quality. The investigator will then create new probabilistic models and guided artificial intelligence frameworks designed to maximize output quality under strict power and memory constraints. The technical scope includes the creation of an integrated, shared computing module for adaptive resource allocation. A case study on urban planning, conducted in partnership with small businesses, will demonstrate the practical application of these resource-aware frameworks. By bridging the gap between computing efficiency and human perception, the award provides foundational data and tools for a new class of human-centered visual generation technologies. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria. NSF Award ID: 2541773 | Program: 01002627DB NSF RESEARCH & RELATED ACTIVIT,01002930DB NSF RESEARCH & RELATED ACTIVIT,01003031DB NSF RESEARCH & RELATED ACTIVIT | Principal Investigator: Qi Sun | Institution: New York University, NEW YORK, NY | Award Amount: $351,857 View on NSF Award Search: https://www.nsf.gov/awardsearch/show-award/?AWD_ID=2541773 View on Research.gov: https://www.research.gov/awardapi-service/v1/awards/2541773.html