CAREER:Toward Generative Cinematography with Controllable Video Synthesis

Video is rapidly becoming the dominant medium for communication, education, and entertainment, yet creating high-quality video content remains costly and technically demanding. Current artificial intelligence models can generate impressive videos from text descriptions, but they operate as black-boxes, offering users little control over the content, motion or camera perspective. This project aims to develop a new paradigm for controllable video generation by unlocking the rich but hidden capabilities embedded within powerful pre-trained video generation models without requiring any additional training or curated data. The resulting framework will give users director-like control over visual content, motion, and camera dynamics and will make powerful creative technologies broadly accessible. Integrated education and outreach activities will broaden understanding of generative artificial intelligence across audiences from elementary school students to creative industry professionals. The research is organized into three interconnected thrusts. Thrust 1 establishes a foundational understanding of text-to-video models by investigating how their internal mechanisms, including noise, attention, and positional embeddings influence generation. Thrust 2 builds on these insights to develop training-free methods for controlling content, motion, and camera viewpoints. Thrust 3 integrates these capabilities into a multi-agent collaboration framework that autonomously reasons, plans, and iteratively refines video synthesis, with built-in self-repair capabilities. The project will produce open-source tools, datasets, and interactive demonstrations, contributing both theoretical understanding of latent representations in video diffusion models and practical advances toward generative cinematography as a universally accessible medium. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria. NSF Award ID: 2543524 | Program: 01003031DB NSF RESEARCH & RELATED ACTIVIT,01002627DB NSF RESEARCH & RELATED ACTIVIT,01002930DB NSF RESEARCH & RELATED ACTIVIT | Principal Investigator: Pinar Yanardag Delul | Institution: Virginia Polytechnic Institute and State University, BLACKSBURG, VA | Award Amount: $325,054 View on NSF Award Search: https://www.nsf.gov/awardsearch/show-award/?AWD_ID=2543524 View on Research.gov: https://www.research.gov/awardapi-service/v1/awards/2543524.html