CAREER: Foundations of Cloud Robotics - Where Neural Networks Meet Physical Networks

Fleets of networked robots are increasingly being deployed in warehouses, hospitals, and transportation systems for industrial automation and assistive robotics. These fleets rely on massive volumes of continuously generated multimodal sensory data that has to be processed in real time for perception and control. There is an inherent trade-off between lightweight but imprecise models that can perform routine inference on the device itself under limited computation and power constraints, and more performant models that have to run in the cloud and are subject to latency, bandwidth, and privacy constraints. This project will study principled ways to address this trade-off in a dynamic, adaptive manner. This project develops the algorithmic foundations for cloud robotic systems to continuously learn and adapt under network constraints. By enabling robots to query the cloud only when required, efficiently share task-relevant data, and robustly adapt to changing network conditions, the project aims to improve the scalability, robustness, and safety of robotic fleets. The project is organized in three inter-related thrusts. The first thrust will develop decision-theoretic frameworks that gracefully trade off between local edge computation and cloud inference under latency and bandwidth constraints while optimizing task accuracy and compute costs. The second thrust targets novel distributed data collection and multimodal retrieval algorithms that identify informative and safety-critical edge cases from petabytes of robotic fleet data under bandwidth and storage constraints. The third thrust focuses on network-aware representation learning methods that compress multimodal sensor streams into concise task-relevant representations, thereby ensuring privacy. The project will also develop multimodal generative models and digital-twin frameworks that synthesize realistic network latency and throughput traces for stress-testing networked robotic systems under rare failures and adversarial conditions. Collectively, project thrusts will establish the foundations for lifelong learning in cloud robotic systems. Broader impacts include K-12 outreach and developing new open source cloud robotics courses and software. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria. NSF Award ID: 2542143 | Program: 01002627DB NSF RESEARCH & RELATED ACTIVIT | Principal Investigator: Sandeep Chinchali | Institution: University of Texas at Austin, AUSTIN, TX | Award Amount: $500,000 View on NSF Award Search: https://www.nsf.gov/awardsearch/show-award/?AWD_ID=2542143 View on Research.gov: https://www.research.gov/awardapi-service/v1/awards/2542143.html