Large-scale artificial intelligence using radiomics, deep learning, and physics-based generative modeling to improve clinical outcomes in nuclear medicine

/Abstract Cancer is the second leading cause of death in the United States where delays in diagnosis and treatment lead to increased mortality and advanced-stage disease. Developing artificial intelligence (AI) and deep learning (DL) approaches for the automatic characterization of malignant disease can facilitate the early detection, diagnosis, prognosis, and treatment of cancer. Radiomics and DL approaches extract quantitative information and visual features from radiological data to glean insights into a patient’s disease. Traditional radiomics approaches suffer from reproducibility issues due to small dataset sizes and differences in imaging scanners, reconstruction methods, and operator variability in regions of interest segmentation. DL methods require training on large datasets with annotated ground truth, which is difficult to obtain due to the limited availability of physician-defined annotations and histopathological ground truth. Radiomics and DL methods are often trained on datasets that encompass a specific malignancy, which additionally limits their generalizability and overall utility. Nuclear medicine imaging modalities provide important functional information regarding radiotracer uptake in benign and malignant pathologies that can help inform diagnosis and treatment. There is a significant unmet need to develop research and clinical tools that address the challenges of enabling large-scale AI-based pipelines in nuclear medicine. Aim 1 will build a large database of clinical positron emission tomography (PET)/computed tomography (CT) images with physician-annotated ground truth. Aim 2 will develop a physics-guided deep generative modeling approach to generate realistic simulated PET/CT data with known ground truth. Aim 3 will quantify the robustness of radiomic features using both simulated and clinical PET/CT data. Aim 4 will develop and validate a simulation-based transfer learning approach on automated lesion detection, segmentation, and classification tasks. Aim 5 will develop and validate a multipronged approach that combines robust radiomics, DL, and ensemble meta-learning to predict clinical outcomes from PET/CT images of patients with cancer. In the K99 training phase of this grant, Dr. Kevin H. Leung will conduct the proposed research under the guidance of Dr. Martin G. Pomper with the support of outstanding advisory committee members with extensive expertise in radiology, oncology, PET, CT, and medical imaging physics. The major objective of the mentored research phase is to create a large clinical PET/CT database encompassing a wide range of cancers and to develop a physics- guided approach to generate realistic simulated PET/CT data that reflect clinical population-level characteristics. The technology developed from the K99 phase will be expanded in the independent R00 phase into a generalized platform that will enable large-scale AI in nuclear medicine for a wide range of medical image analysis tasks. The rich resources and strong collaborations available at Johns Hopkins provide an ideal training environment that is completely supportive of the proposed research and the academic advancement of Dr. Leung. Project Number: 4R00CA287045-03 | Fiscal Year: 2026 | NIH Institute/Center: National Cancer Institute (NCI) | Principal Investigator: Kevin Leung | Institution: JOHNS HOPKINS UNIVERSITY, BALTIMORE, MD | Award Amount: $249,000 | Activity Code: R00 | Study Section: ZCA1-RTRB-U(O1) View on NIH RePORTER: https://reporter.nih.gov/project-details/11457139