CAREER: From Learning to Forgetting: Linguistic Complexity, Learning Dynamics, and Robustness in Neural Language Models

Artificial intelligence systems that generate text now influence how people search for information, learn new skills, obtain health advice, and communicate online. Yet these systems can behave in ways that are hard to predict. They can copy misleading patterns from their training data, produce text that is too complex for a reader's needs, and retain private, outdated, or incorrect content. This project studies how these systems learn, keep, and forget language patterns over time, and uses that knowledge to build systems that are easier to control and safer to use. The results can help create text at appropriate reading levels for second language learners, patients reading health information, and people with communication or cognitive challenges. The project also develops methods to remove harmful patterns without weakening a model's general ability to generate useful text. The project advances reliable artificial intelligence, improves access to understandable information, and trains students through coursework, mentoring, freely available tools, and interdisciplinary workshops to support science and public well-being. The project develops a linguistically grounded framework for robust and interpretable neural language models. It creates methods for controlled text generation and paraphrasing that allow models to follow user-defined lexical, syntactic, and discourse constraints. These methods combine instruction tuning, explicit control signals, multi-objective optimization, and iterative test time refinement to satisfy several user-defined constraints simultaneously. The project also studies how language models learn over time during training by measuring performance on words and text spans with linguistic annotations. This creates learning timelines that can show when specific language properties are learned, help better align training data with model abilities, and support more efficient training. In addition, the project develops methods to remove harmful patterns while preserving overall language ability. These methods use input perturbations to expose shortcut behavior, contrastive learning to reduce reliance on spurious cues, and smooth low-loss update paths to control forgetting. The methods will be evaluated on benchmark tasks for controllability, linguistic generalization, paraphrasing, and forgetting. Expected outcomes include new models, diagnostic methods, data resources, and publicly available tools for more reliable language technologies. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria. NSF Award ID: 2541273 | Program: 01002930DB NSF RESEARCH & RELATED ACTIVIT,01002627DB NSF RESEARCH & RELATED ACTIVIT,01003031DB NSF RESEARCH & RELATED ACTIVIT | Principal Investigator: Hadi Amiri | Institution: University of Massachusetts Lowell, LOWELL, MA | Award Amount: $341,598 View on NSF Award Search: https://www.nsf.gov/awardsearch/show-award/?AWD_ID=2541273 View on Research.gov: https://www.research.gov/awardapi-service/v1/awards/2541273.html