Background/Objectives: Lung cancer is a devastating disease with the highest mortality rate among cancer types. Over 60% of non-small cell lung cancer (NSCLC) patients, accounting for 87% of lung cancer diagnoses, require radiation therapy. Rapid treatment initiation significantly increases the patient’s survival rate and reduces the mortality rate. Accurate tumor segmentation is a critical step in diagnosing and treating NSCLC. Manual segmentation is time- and labor-consuming and causes delays in treatment initiation. Although many lung nodule detection methods, including deep learning-based models, have been proposed. Most of these methods still have a long-standing problem of high false positives (FPs). Methods: Here, we developed an electronic health record (EHR)-guided lung tumor auto-segmentation called EXACT-Net (EHR-enhanced eXACtitude in Tumor segmentation), where the extracted information from EHRs using a pre-trained large language model (LLM) was used to remove the FPs and keep the TP nodules only. Results: The auto-segmentation model was trained on NSCLC patients’ computed tomography (CT), and the pre-trained LLM was used with the zero-shot learning approach. Our approach resulted in a 250% boost in successful nodule detection using the data from ten NSCLC patients treated in our institution. Conclusions: We demonstrated that combining vision-language information in EXACT-Net multi-modal AI framework greatly enhances the performance of vision only models, paving the road to multimodal AI framework for medical image processing.
背景/目的：肺癌是一种具有毁灭性的疾病，在各类癌症中死亡率最高。非小细胞肺癌（NSCLC）占肺癌诊断的87%，其中超过60%的患者需要接受放射治疗。快速启动治疗能显著提高患者生存率并降低死亡率。精确的肿瘤分割是诊断和治疗NSCLC的关键步骤。人工分割耗时耗力，常导致治疗启动延迟。尽管已有许多肺结节检测方法被提出，包括基于深度学习的模型，但这些方法大多仍存在假阳性率高的长期问题。方法：本研究开发了一种电子健康记录（EHR）引导的肺肿瘤自动分割系统EXACT-Net（基于EHR增强的肿瘤分割精确网络），该系统通过预训练大语言模型（LLM）提取EHR信息，用于消除假阳性结节并仅保留真阳性结节。结果：自动分割模型基于NSCLC患者的计算机断层扫描（CT）数据进行训练，预训练LLM采用零样本学习方法。通过对本机构治疗的10例NSCLC患者数据进行分析，该方法使结节检测成功率提升250%。结论：研究表明，在EXACT-Net多模态人工智能框架中融合视觉与语言信息，能显著提升纯视觉模型的性能，为医学图像处理的多模态人工智能框架发展开辟了新路径。
肿瘤(癌症)患者之家