CAREER: Toward Scalable and Resilient Collaborative Inference for Edge Intelligence through Network-Aware Design

Collaborative inference at the network edge enables low-latency and privacy-sensitive artificial intelligence (AI) services without relying on remote cloud infrastructure. Distributing increasingly complex neural network models across nearby edge devices enables the pooling of their compute and memory resources to perform inference beyond the capability of any single device. Existing collaborative inference approaches rely on centralized or static control under assumptions of stable network connectivity, leading to inefficient resource use and degraded inference performance when network or system conditions change. This project reconceptualizes the wireless network not merely as a communication medium, but as a coordination substrate to orchestrate model execution and resource allocation for efficient, scalable, and resilient collaborative inference. The resulting advances will support emerging applications such as mobile health, distributed robotics, and intelligent transportation systems. The project also integrates research into teaching through new courses with hands-on learning modules on edge intelligence and distributed AI. It further strengthens an existing mentoring pipeline spanning K-12 outreach, undergraduate research participation, and graduate training, preparing students for future careers across AI, systems, and networking. The project advances the scientific foundations of network-aware collaborative inference at the intersection of networking, distributed systems, and AI. Low-cost decentralized awareness of network and device conditions, maintained and shared by edge devices, serves as a unifying foundation across the three research thrusts. Thrust 1 develops a decentralized inference framework that jointly optimizes model partitioning and resource allocation under partial observability for efficient and scalable general-purpose inference. The growing demand for generative inference with billions of parameters and autoregressive decoding introduces new system challenges. Thrust 2 therefore develops pipeline parallelism strategies for coordinating pipeline execution and inter-edge communication to improve the efficiency of collaborative inference under limited bandwidth and constrained edge resources. Thrust 3 develops model-agnostic runtime coordination mechanisms that sustain both general-purpose and generative inference performance under connectivity disruptions and device failures. The research combines algorithm design, systems prototyping, and experimental evaluation in realistic edge environments. Together, these advances will help shape future intelligent networked systems that integrate communication, computing, and AI. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria. NSF Award ID: 2544108 | Program: 01002627DB NSF RESEARCH & RELATED ACTIVIT,01003031DB NSF RESEARCH & RELATED ACTIVIT,01002930DB NSF RESEARCH & RELATED ACTIVIT | Principal Investigator: Xiaonan Zhang | Institution: Florida State University, TALLAHASSEE, FL | Award Amount: $374,823 View on NSF Award Search: https://www.nsf.gov/awardsearch/show-award/?AWD_ID=2544108 View on Research.gov: https://www.research.gov/awardapi-service/v1/awards/2544108.html