CAREER: Enabling Ultra-Low-Cost Cloud-Free Data Processing at the Edge

The amount of data processed by AI-based analytics has grown exponentially year over year. Unfortunately, this has unveiled a critical deficiency of contemporary computers: they physically separate their computation chips from their data storage chips, requiring each piece of data to move between the two whenever it needs to be processed. This separation wastes an increasing amount of energy and time on data movement, often 100 to 1000 times more than the computation itself, hindering the performance, efficiency, and potential uses of computers. For mobile computers such as smartphones, small drones, and sensors, the waste forces them to offload most analytics to the cloud. Recent breakthroughs in computer hardware have realized a near-60-year vision to unite computation and storage onto the same chip, potentially eliminating most data movement waste. If fully realized, this new hardware could unleash the next revolution in mobile computing, with ultra-low-cost devices that no longer need the cloud or an Internet connection to perform AI data analytics in mobile computers. This project addresses the major challenges preventing this realization, by jointly designing the hardware and software foundations required to support and integrate these new chips. The research produced by this project can enable many new uses of computers, from automated crop monitoring to all-day extended reality gaming to rural healthcare to infrastructure anomaly detection. All research findings will be publicly disseminated through conference publications, openly available software tools, and a project website. The project will introduce new university classes, K-12 programs, and public outreach campaigns to introduce learners to hardware/software co-design skills that are critical to train the next generation of computer engineers. Specifically, this project will make cloud-free edge AI analytics a reality by evolving today's processing-using-memory (PUM; a.k.a. in-memory computing) accelerators into standalone, scalable systems. PUM accelerators use electrical interactions inside memory arrays to perform approximate analog multiplies or precise general-purpose Boolean operations. While prior research has developed PUM datapaths that can execute a limited range of parallel microkernels, the research team will use hardware/software co-design to tackle three critical challenges for Boolean-PUM-based edge AI analytics. First, the team will design cross-stack abstractions that enable in-PUM execution of thread-based software and allow PUM to execute applications without CPU assistance. Second, the team will design data protection mechanisms that enable PUM virtual memory and address security and privacy concerns with today's PUM accelerators. Third, the team will design a meta-manager that allows system developers to cohesively coordinate edge analytics across many PUM chips in a scalable manner. The work will introduce several firsts, including end-to-end application execution in PUM without a CPU, a software stack for general-purpose PUM, the unification of memory allocation and thread scheduling for data-centric systems, generation-based virtual memory management, and multi-PUM-chip coordination. Together, these solutions can enable the deployment of standalone cloud-free PUM edge platforms that deliver several-orders-of-magnitude higher energy efficiency while significantly reducing programming complexity; and catalyze new research areas in PUM systems, programming languages, and ubiquitous edge analytics. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria. NSF Award ID: 2543446 | Program: 01002627DB NSF RESEARCH & RELATED ACTIVIT,01003031DB NSF RESEARCH & RELATED ACTIVIT,01002930DB NSF RESEARCH & RELATED ACTIVIT | Principal Investigator: Saugata Ghose | Institution: University of Illinois at Urbana-Champaign, URBANA, IL | Award Amount: $420,719 View on NSF Award Search: https://www.nsf.gov/awardsearch/show-award/?AWD_ID=2543446 View on Research.gov: https://www.research.gov/awardapi-service/v1/awards/2543446.html