CAREER: Adaptive Knowledge Synthesis for Language Model Reasoning

Large Language Models (LLMs) have shown impressive performance on pure reasoning tasks like math questions and logic puzzles. However, Artificial Intelligent (AI) systems with such LLMs still struggle with complex tasks that require finding and combining information from multiple external sources. For instance, answering a complicated legal or scientific question often requires a step-by-step investigation where the answer to one question determines what to search for next. While current AI systems can perform multiple searches, they typically rely on a single, general-purpose model that struggles to form a structured plan or adapt its expertise to different phases when investigating a problem in depth. This project addresses this critical gap by creating a new way for AI systems to actively seek out and synthesize knowledge. By empowering models to formulate step-by-step research plans and adapt to each part of a problem, this project promotes the progress of science and advances national prosperity through the creation of highly reliable and transparent decision-making tools. These new capabilities will directly benefit evidence-based fields such as legal analysis and scientific discovery. In addition to these technological benefits, the project supports education by creating new university courses that teach students how to critically analyze AI systems. The investigator will also lead hands-on outreach activities for local school students to inspire the next generation of researchers. The technical goal of this award is to establish a proactive, efficient, and transparent reasoning framework for knowledge-intensive tasks. The research activities are organized into three integrated thrusts. The first thrust develops the Adaptive Knowledge Synthesis framework, which trains LLMs to formulate structured reasoning plans through hypothesis-driven decomposition. Within this framework, a conductor model dynamically reconfigures a base LLM to create specialized experts tailored to specific sub-tasks, using confidence-guided routing to synthesize a final answer. Because this multi-step process generates long dialogue histories and extensive intermediate outputs, the second thrust develops scalable inference methods to reduce computational costs. On the input side, an activation-level integration will be introduced to effectively process long contexts. On the output side, the research team will develop compressed reasoning techniques and sample-efficient test-time scaling methods to streamline generation. Finally, the third thrust establishes a new generation of evaluation frameworks to rigorously measure both procedural correctness and computational efficiency. This involves building controllable benchmarks with systematically varying difficulty, robustness tests featuring multi-hop contradictions, and efficiency-aware metrics that evaluate the trade-off between reasoning accuracy and computational cost. All resulting models, algorithms, and evaluation frameworks will be released as open-source resources to benefit the broader research community. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria. NSF Award ID: 2541822 | Program: 01002627DB NSF RESEARCH & RELATED ACTIVIT,01002930DB NSF RESEARCH & RELATED ACTIVIT,01003031DB NSF RESEARCH & RELATED ACTIVIT | Principal Investigator: Jiaxin Huang | Institution: Washington University, SAINT LOUIS, MO | Award Amount: $417,565 View on NSF Award Search: https://www.nsf.gov/awardsearch/show-award/?AWD_ID=2541822 View on Research.gov: https://www.research.gov/awardapi-service/v1/awards/2541822.html