CAREER: A Networked 4D Architecture and Systems Foundation for Spatial Intelligence

Spatial intelligence enables computing systems to perceive and interact with the physical world. Emerging spatial intelligence technologies such as smart glasses and extended reality systems are beginning to bring digital information directly into everyday physical spaces over the consumer Internet, helping people perform complex tasks in education, healthcare, manufacturing, and logistics more efficiently. Strategic adoption of these technologies could yield billions of dollars in savings. However, today’s Internet is designed primarily for 2D images and video rather than for rich, dynamic 3D, i.e., 4D spatial information. As a result, current systems struggle to efficiently send 4D spatial data on the Internet, limiting the reliability and scalability of these technologies. This research will enable smart glasses to understand and communicate 4D environments in real time, making immersive technologies more practical, accessible, and reliable. The project also expands educational opportunities by introducing students to spatial computing through hands-on experiments, open-source tools, and outreach programs that engage learners from high school through graduate school. The project contributes to workforce development and strengthens national leadership in emerging computing technologies by building accessible platforms and training the next generation of engineers and researchers. This award develops a systems and networking foundation for spatial intelligence by advancing methods for representing, transmitting, and programming 4D spatial data in resource-constrained devices such as smart glasses. The research introduces a hybrid spatial representation that combines structured-mesh geometry with Gaussian-splat-based appearance models to enable real-time spatial perception, rendering, and analytics such as localization and tracking. Building on this representation, the project develops compression and streaming techniques that exploit temporal redundancy in dynamic meshes and allocate network bandwidth efficiently between geometry and texture components. These techniques include 4D mesh compression methods, predictive bitrate control models, and adaptive streaming strategies that jointly optimize visual fidelity and network performance. The project further investigates networked spatial systems through an open hardware and software platform for experimental smart glasses. The platform exposes tunable components across the sensing, networking, and computational layers, enabling researchers and developers to study cross-layer trade-offs among latency, energy consumption, and visual quality. The resulting system will support reproducible research in networked spatial computing and provide new tools for evaluating real-time perception, streaming, and multi-user spatial applications. Additionally, the award will lead to educational activities, including the development of hands-on courses and virtual laboratory environments that use spatial computing systems to illustrate complex systems concepts. The research outcomes will allow students to observe and interact with abstract computing processes in real time, creating new opportunities for experiential learning in networking and distributed systems. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria. NSF Award ID: 2544317 | Program: 01003031DB NSF RESEARCH & RELATED ACTIVIT,01002930DB NSF RESEARCH & RELATED ACTIVIT,01002627DB NSF RESEARCH & RELATED ACTIVIT | Principal Investigator: Mallesham Dasari | Institution: Northeastern University, BOSTON, MA | Award Amount: $361,484 View on NSF Award Search: https://www.nsf.gov/awardsearch/show-award/?AWD_ID=2544317 View on Research.gov: https://www.research.gov/awardapi-service/v1/awards/2544317.html