Collaborative Research: Elements: Infrastructure for High-Performance Distributed Sparse Tensor Computations

Tensors are fundamental data structures underpinning applications in scientific/engineering computing and machine learning. High-dimensional sparse tensors contain many zero elements and thus require specialized data representations and optimized algorithms. Robust support for high-performance sparse tensor computations is currently lacking, for example, for the challenging sparse tensor operator graphs arising in quantum chemistry. Both within a single machine and for distributed execution across multiple machines, there is a pressing need for software infrastructure that accelerates software development and the scale/performance of distributed scientific computations on sparse tensors. This project will build such an infrastructure to help scientists, especially in the fields of Quantum Chemistry and Machine Learning, to achieve (1) high performance, (2) reduced effort for software development, and (3) performance portability for distributed sparse tensor computations. The project makes contributions along multiple directions: (1) Multi-Level Intermediate Representation (MLIR) integration: integration with the popular MLIR compiler infrastructure, to enhance sustainability and dissemination; (2) data structures and algorithms for sparse tensors: novel hash-based data representations for sparse tensors, together with corresponding efficient parallel and adaptive algorithms for tensor operators; (3) optimizations for tensor operator graphs: new operator fusion optimizations for graphs of tensor operators, to reduce memory and increase performance; (4) distributed tensors: data structures and efficient operations to enable high-productivity development of distributed sparse tensor algorithms, together with compiler support to automatically generate implementations of distributed sparse tensor operators with minimized data movement costs; (5) engagement with domain scientists to achieve and sustain infrastructure impact. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria. NSF Award ID: 2609203 | Program: 01002627DB NSF RESEARCH & RELATED ACTIVIT | Principal Investigator: Atanas Rountev | Institution: OHIO STATE UNIVERSITY, THE, COLUMBUS, OH | Award Amount: $299,767 View on NSF Award Search: https://www.nsf.gov/awardsearch/show-award/?AWD_ID=2609203 View on Research.gov: https://www.research.gov/awardapi-service/v1/awards/2609203.html