CAREER: Systems for Finely Interleaved Programs

Hardware vendors are designing special-purpose accelerators to offload more work from general-purpose processors in order to improve the performance and efficiency of applications such as artificial intelligence. However, it is challenging for applications to take advantage of these accelerators today because applications typically interleave common operations that are amenable to hardware acceleration with application-specific logic that is difficult to offload. These interleaved operations can occur at very fine granularity, inhibiting efficient and scalable execution. In addition, the diversity of available accelerators makes it hard to write programs that leverage them. This project aims to understand how existing applications interleave operations that are and are not amenable to hardware acceleration, to build systems that enable efficient and scalable interleaving at fine granularity, and to create frameworks that facilitate writing applications that leverage fine-grained accelerators. This project will enable a new computing paradigm of finely interleaved programs, or programs that interleave compute across general-purpose processors and specialized accelerators at microsecond timescales. In its first thrust, this project will anticipate how finely interleaved programs will be structured, identifying candidate operations for offloading and modeling the control flow and data flow between general-purpose processors and accelerators. Second, this project will design a runtime system to execute finely interleaved programs efficiently and scalably, bypassing the processor when accelerated operations can be chained directly. In the third thrust, this project will design a programming model and compiler that enable fine-grained offloads with minimal developer effort to rewrite applications. In combination, these three thrusts will make it possible to understand, write, and execute finely interleaved programs. This project will make it easier and more efficient to offload tasks to emerging accelerators, directly reducing hardware requirements, energy use, costs, and developer effort. By making fine-grained accelerators more usable, this work will also impact chip designers, inspiring a wave of increasingly fine-grained accelerators, which will further improve efficiency. Finally, this project will build an accelerator emulation platform that researchers and students can use to understand and test new techniques for supporting finely interleaved programs. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria. NSF Award ID: 2543083 | Program: 01002627DB NSF RESEARCH & RELATED ACTIVIT,01002930DB NSF RESEARCH & RELATED ACTIVIT,01003031DB NSF RESEARCH & RELATED ACTIVIT | Principal Investigator: Amy Ousterhout | Institution: University of California-San Diego, LA JOLLA, CA | Award Amount: $507,378 View on NSF Award Search: https://www.nsf.gov/awardsearch/show-award/?AWD_ID=2543083 View on Research.gov: https://www.research.gov/awardapi-service/v1/awards/2543083.html