CAREER: Grid-Based Code Representations to Support Structured, Accessible Programming Environments and Education

Computer programming has evolved from low-level machine languages to modern high-level languages such as Python. Tools for coding programs in these high-level languages often use visual elements such as parentheses, spacing, and highlighting to help people understand and create the hierarchical structures that underlie most modern programming languages. Programming environments also use visual cues, such as highlighting changes in the program’s data as it runs, to help programmers find and fix errors. For blind and low-vision (BLV) individuals who rely on assistive technologies such as screen readers, much of this information is lost when visual representations of code and program behavior are converted into a linear stream of synthesized speech. This project’s goal is to create new ways to represent programs that allow BLV individuals to fully participate in programming while using their existing accessibility tools and practices. The project team will make the new insights, tools, and educational materials widely available, which will increase BLV individuals’ ability to participate in the many jobs and other activities where programming is an important skill. The project is structured around two main research aims, each of which develops a key new representation of programs and behavior. The first aim, Grid-Coding, transforms source code into an explicit two-dimensional grid in which rows represent code lines, columns represent scope, and padding becomes meaningful structure. This representation supports non-traditional navigation strategies, including right-to-left, bottom-up, and level-based traversal, so that learners can inspect hierarchy through multiple pathways, recognize recurring code shapes, identify code smells such as deep nesting or overly long methods, and turn padding cells into active learning surfaces for hints, annotations, and locally hosted AI assistance. The second aim, Grid-Time Volume, transforms debugging from a transient visual event into a persistent navigable space-time volume that stores program states over time. Learners will move backward and forward through execution, inspect memory layout and call structure, and monitor multiple variables through natural soundscapes inspired by birdsong in a forest, where changes in loudness, rhythm, and spatial position convey runtime behavior. Through participatory design, controlled experiments, and longitudinal studies with blind and low-vision learners, the project will examine how these representations affect code comprehension, code construction, debugging, mental model formation, cognitive load, and retention. The education plan will translate these advances into open-source programming environment extensions, two fully accessible video-based courses, modules for undergraduate curricula, and community-based summer programs. The work serves the national interest by widening access to computing education, supporting a larger technical workforce, and producing open resources that can strengthen instruction in mixed-ability classrooms and self-directed learning. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria. NSF Award ID: 2543660 | Program: 01002930DB NSF RESEARCH & RELATED ACTIVIT,01002627DB NSF RESEARCH & RELATED ACTIVIT,01003031DB NSF RESEARCH & RELATED ACTIVIT | Principal Investigator: Syed Billah | Institution: Pennsylvania State Univ University Park, UNIVERSITY PARK, PA | Award Amount: $424,193 View on NSF Award Search: https://www.nsf.gov/awardsearch/show-award/?AWD_ID=2543660 View on Research.gov: https://www.research.gov/awardapi-service/v1/awards/2543660.html