Menu
Menu
Lung cancer is a leading cause of cancer-related deaths worldwide, presenting substantial clinical and imaging challenges, especially in its most advanced stages, such as stage IV NSCLC.
Medical imaging, such as computed tomography (CT) scans and magnetic resonance imaging (MRI), plays a crucial role throughout the lung cancer patients’ care pathway, from initial diagnosis, through treatment response assessment, to longitudinal monitoring.
Accurate tumor segmentation remains a noteworthy gap in clinical workflows, being particularly difficult in patients with stage IV NSCLC, due to heterogeneous tumor morphology and the presence of associated comorbidities and conditions caused by the advanced state of their disease (e.g., atelectasis, and pleural effusion).
As such, there’s a clear growing need for developing robust and reproducible radiomics segmentation tools for advancing personalized medicine in NSCLC and providing improved patient care.
This study addresses the persistent obstacle of developing high-performance tumor segmentation using artificial intelligence (AI), in highly heterogeneous multicentric 3D CT scans of metastatic NSCLC (mNSCLC), with an emphasis on model trustworthiness via uncertainty quantification.
To conduct this study, Dedeken et al. used a curated subset of data from the DEEP-Lung-IV (DLIV) study (NCT04994795), comprising 387 stage IV NSCLC patients from 13 European centers. These patients were treated either with a combination of chemotherapy and immunotherapy (study cohort B) or with chemotherapy alone (study cohort C).
The images were acquired using different device models and acquisition protocols across centers, increasing the heterogeneity of the dataset, which reflected the variety found in real-world clinical practice.
All images were annotated by expert radiologists and pre-processed (Figure 1), to ensure consistency, before running AI models to delineate the tumors.
The team evaluated three segmentation models widely used in literature: U-Net, Attention U-Net, and UNEt Transformed (UNETR). They also introduced a “confidence score” to help identify when the AI model might be unsure, especially in more complex cases like small or hard-to-segment tumors. This score helps flag the uncertain cases for further review, ensuring more reliable downstream analyses.
1. Segmentation Performance
The best performing AI model was the Attention U-Net, which was trained with a special method to boost accuracy. It correctly segmented lung tumors with a score of 0.76 (±0.20).
This performance surpassed U-Net (0.66 ±0.23) and UNETR (0.60 ±0.23), demonstrating that the model is reliable across different patient subsets and types of settings.
As expected, the model showed lower scores in cases with small tumor volumes or complex scenarios, such as atelectasis and pleural effusion.
Additionally, the researchers found that using a narrower imaging range focused on the abdomen yielded better precision in solid tumor segmentation, critical for advanced NSCLC imaging needs.
2. Confidence Score Reliability
The developed confidence score, following a Deep Ensembles approach, showed how certain the AI segmentation model was about each result. This score demonstrated high accuracy, closely matching the actual performance, with a strong correlation of 0.86.
Low-confidence scores were associated to known complex cases (e.g., small tumor volumes, pleural effusion, atelectasis), showing that the system can automatically flag tricky images for additional expert manual verification.
A Monte Carlo Dropout (MCDO) method was also tested for computing uncertainty. Although the results obtained were satisfying (correlation of 0.77), the Deep Ensembles approach produced more reliable results.
The development of accurate tumor segmentation from CT scans is essential for extracting reliable imaging biomarkers that support clinical decision-making and personalized treatment strategies in advanced NSCLC.
This study by Dedeken et al. aligns with these growing needs and demonstrates that trustworthy deep-learning-based tumor segmentation can effectively address the complexity of segmenting stage IV NSCLC CT scans, even using diversified, heterogeneous, and multicentric datasets.
By running extensive experiments, this study showed that:
Next steps include testing the developed algorithms on a larger and more diversified group of patients leveraging the DEEP-Lung-IV clinical study; and assessing them with stage III NSCLC patients undergoing radiotherapy and chemotherapy to evaluate the generalizability of the segmentation model.
Overall, this work lays the foundation for robust and explainable radiomics algorithms, offering insights and tools to accelerate the integration of AI in medical imaging workflows.
This study was led by Sacha Dedeken, Pierre-Henri Conze, and Dimitris Visvikis from the Laboratory of Medical Information Processing (LaTIM), in collaboration with SOPHiA GENETICS.
LaTIM is a joint research laboratory (UMR) of Inserm (French National Institute of Health and Medical Research), the University of West Brittany (UBO), and IMT Atlantique, associated with the CHRU (University Research Hospital) of Brest.
SOPHiA DDM™ for Radiomics and SOPHiA DDM™ for Multimodal are concepts in development. May not be available for sale.
Just two decades ago, cancer was largely considered an organ-based disease. For example, lung cancer, despite having known histological subtypes, was uniformly treated as a single disease – chemotherapy for all. Today, advances in clinical genomics have transformed lung cancer into a collection of rare diseases defined by a long tail of distinct genomic alterations. Building on this knowledge, targeted therapies have gradually improved patient outcomes for eligible patients. However, significant medical needs remain, particularly regarding overall survival (OS) or achieving durable cures. Most lung cancer patients in first-line therapy do not have a clear molecularly driven cause and almost uniformly receive an immunotherapy-based treatment as a standard of care. Despite massive investments, single-biomarker approaches have failed to reliably predict response to immunotherapy, leaving clinicians unable to determine which patients will benefit most.
To address these challenges, precision medicine must shift from a siloed, single-biomarker approach to a more integrated multimodal approach, combining genomic, imaging, clinical, and biological data. Intuitively, taking a more holistic view of the patient, the tumor, and the host environment should open a stronger window into the biology of health and disease. Yet, realizing the full potential of this approach requires a transformation of the underlying data infrastructure. This includes breaking down silos across data modalities, standardizing and harmonizing datasets and promoting real-world knowledge sharing across institutions. Such transformation is necessary to unlock the power of artificial intelligence (AI) applied to multimodal healthcare data.
In this article, we explore the groundbreaking potential of multimodal AI-driven technology as a key driver of a new era in precision medicine. This paradigm shift promises not only to accelerate innovation but also to improve patient outcomes and expand equitable access to care. With competition intensifying and regulatory scrutiny increasing, biopharma companies must embrace this transformation to stay ahead and reshape the future of precision oncology.
Over the past decade, we have seen a dual revolution in healthcare: the explosion of multiple types of digital health data being produced at scale in clinical routine (e.g., genomics, imaging, electronic health records (EHR) entries) and generational breakthroughs in analytics capabilities (e.g., machine learning, deep learning, foundational models). In theory, this combination should have unlocked the full potential of precision medicine, but in practice, we are arguably still at its Stone Age. One of the root causes of this lies in the fact that healthcare data remains largely fragmented and unharmonized, and the tools to integrate and harness it effectively are often lacking. Additionally, there is no built-in incentive in the ecosystem for large-scale data and insights sharing across institutions, while preserving privacy. The “publish or perish” mantra in academia still tends to encourage data hoarding — consciously or not — further hindering collaboration.
Similarly, on the biopharma side, we see increasing interest and expertise for AI-driven approaches on specific data modalities (e.g., digital pathology, radiology). Yet, these initiatives are still rarely connected in a truly multimodal framework. Proprietary clinical trial databases remain challenging to harmonize and merge together due to heterogeneous patient consents, compliance issues, and required investments.
A fair question to ask is, therefore—is it even worth the trouble?
Although we are just beginning to tap into the capabilities of multimodal AI-driven technology, it is already driving significant advances in our understanding of health and disease. By seamlessly integrating and analyzing diverse data sources, this approach enables a more holistic perspective of complex diseases like cancer, and the patient beyond the disease.
The practical application of AI-powered multimodal technology can help biopharma companies solve complicated biological puzzles and overall optimize the drug development process (Figure 1).
At SOPHiA GENETICS we believe that multimodal AI is no longer an option but a necessity to accelerate precision medicine. Our cloud-based SOPHiA DDM™ Platform seamlessly integrates and standardizes diverse data types into a unified analytical framework, comprising state-of-the-art specialized computational modules for data processing and analysis (e.g., genomics, radiomics), including a dedicated multimodal factory. This engine combines, extracts, and structures complex multimodal data to fuel the development of predictive analytics, delivering actionable insights that empower data-driven decision-making through an intuitive, user-friendly interface (Figure 2).
The potential of this multimodal approach is evident in the initiatives we are leading here at SOPHiA GENETICS. One noteworthy example is the TRIDENT project, presented at ESMO 2024 (Skoulidis et al., 2024). TRIDENT was a retrospective multimodal re-analysis of AstraZeneca’s Phase 3 POSEIDON trial (NCT03164616). AI-powered predictive models of treatment benefit were trained on the totality of the clinical trial data, including clinical, biological, imaging and genomics data, with the intent to identify patient subpopulations that may derive greater benefit from the addition of a CTLA-4 inhibitor on top of a PD-L1 and chemotherapy backbone in first-line non-small cell lung cancer. These models yielded signatures identifying approximately 50% of the trial population in scope that would be predicted to benefit from the addition of CTLA-4 inhibition, with a hazard ratio reduction from 0.88 (95% CI, 0.68-1.12) to 0.56 (95% CI, 0.33-0.97) in the non-squamous histology population (Figure 3). These multimodal signatures are clinically interpretable and can be readily deployed in the real-world setting on the SOPHiA DDM™ Platform for further clinical research.
This proof-of-concept analysis highlights a fundamental observation: traditional methods of analyzing and making sense of existing data, for example, through univariate or multivariate analyses, can leave significant portions of the full picture unseen. In contrast, new multimodal approaches have the potential to reveal insights that would otherwise remain hidden.
The transformational findings from this lung cancer project are not an exception. Similar results were obtained in other cancer types, such as in kidney cancer (Boulenger de Hauteclocque et al., 2023; Margue et al., 2024), and breast cancer (Groheux et al., 2025).
To successfully implement a multimodal AI-driven strategy, it is essential to begin with a clearly defined clinical question. What are you trying to predict or stratify, and why does it matter clinically? What is your endpoint of interest? These considerations will dictate the type, quality and volume of data required for a successful analysis.
A truly multimodal model demands a deep understanding of the signal-to-noise ratio within each individual data modality. For example, in genomics: do you know how the DNA of the tumor was sequenced, which chemistry was used, and on which sequencing platform? How was the variant calling performed, and what are the known limitations? Only with this level of detail can you distinguish meaningful biological signals from background noise. The same applies to radiology, where harmonizing image data across different platforms and reconstruction techniques is essential. Skipping these foundational data preparation steps risks identifying patterns in noise rather than signal.
Once the dataset is well understood and curated, data augmentation can be applied—for instance, through radiomics analysis of 3D medical imaging or pathway analysis of relevant genomic variants.
After augmentation, data aggregation becomes the next critical step. This involves integrating diverse data types into a unified analytical framework. From here, various statistical learning methods can be selected based on the clinical objective. Typically, imputation techniques for missing data need to take place before feature selection, statistical model selection, and calibration. Finally, ensuring analysis interpretability, both at the cohort and individual level is an important step in facilitating discussions with oncologists and other healthcare professionals concerning the methodology and the outcomes of the models (Figure 4). This can be achieved by using traditional machine learning models.
An often overlooked yet crucial step is the deployment of the models in a real-world setting. How will end users interact with the model? How will data be input and managed? What safeguards are in place for data privacy and security, and how will computational infrastructure scale?
At SOPHiA GENETICS, we believe that the cloud-native SOPHiA DDM™ Platform is uniquely positioned to spearhead this movement. Already adopted by over 800 healthcare institutions across more than 70 countries, the platform has securely processed data from more than 2 million patients, ensuring privacy while enabling impactful, AI-powered multimodal insights at scale.
The transition from single-modality to multimodal AI-driven analysis represents a paradigm shift in precision medicine. Organizations that successfully integrate diverse data modalities and multimodal technology will be best positioned to drive better patient outcomes and maximize drug development success.
Realizing this potential, however, demands more than technical innovation. It demands systemic transformation across the overall healthcare landscape — from evolving regulatory frameworks for multimodal CDx, to updated reimbursement models, standardized deployment practices— inclusive of post-market surveillance—, and greater education for both clinicians and patients.
For biopharma companies, the path forward is clear:
About twenty years ago, cancer was still considered an organ disease. Looking back today, this may look like distant, medieval times. Twenty years from now, new generations of life sciences professionals may look at 2025 in a disturbingly similar way. The multimodal revolution is only now getting started.
Written by Philippe Menu, MD, PhD, MBA - EVP, Chief Medical Officer & Chief Product Officer, SOPHiA GENETICS
References
Skoulidis F et al. 1325P TRIDENT: Machine learning (ML) multimodal signatures to identify patients that would benefit most from Tremelimumab (T) addition to durvalumab (D) + chemotherapy (CT) with data from the POSEIDON trial. Ann. Oncol. 35, S842–S843 (2024).
Boulenger de Hauteclocque A et al. Machine-learning approach for prediction of pT3a upstaging and outcomes of localizaed renal cell carcinoma (UroCCR-15). BJU Int. 2023; 132(2):160-169. doi: 10.1111/bju.15959.
Margue G et al. UroPredict: Machine learning model on real-world data for prediction of kidney cancer recurrence (UroCCR-120). NPJ Precis Oncol. 2024; 8(1):45. doi: 10.1038/s41698-024-00532-x.
Groheux D et al. FDG-PET/CT and Multimodal Machine Learning Model Prediction of Pathological Complete Response to Neoadjuvant Chemotherapy in Triple-Negative Breast Cancer. Cancers. 2025; 17(7):1249. doi: 10.3390/cancers17071249.
Radiation pneumonitis (RP) is a significant and relatively common complication of (chemo) radiotherapy (RT) in the treatment of locally advanced non-small cell lung cancer (LA-NSCLC).
This study investigates clinical, dosimetric, and radiomic features predictive of lung toxicity, specifically grade (G)≥2 RP, in patients undergoing (chemo)RT for LA-NSCLC, with the aim to build a predictive model to estimate its occurrence.
The researchers conducted a retrospective multicenter analysis of 153 patients treated with (chemo)RT, between 2015 and 2019, to identify key risk factors. Baseline CT scans were segmented to extract radiomic features from the lungs and the tumor, and integrate them with clinical and dosimetric features.
The study employed a machine learning (ML) approach using logistic regression and random forest models to develop predictive models for RP occurrence in this patient population.
The clinical and dosimetric risk factors linked to an increased RP risk included high initial hemoglobin levels, older age, low Tiffeneau ratio (FEV1/VC), decreased initial platelet count, dosimetric factors (mean dose to lungs, lung V20Gy and V13Gy), and the use of adjuvant durvalumab.
Seven radiomic features related to intensity distribution and texture were significantly associated with RP risk.
The developed ML-based model (random forest) integrating clinical, dosimetric and radiomic data achieved the best performance with an AUC = 0.72 (95% CI [0.63-0.80]), outperforming models based on combined clinical and dosimetric data (AUC = 0.64), or on radiomic data alone (AUC=0.64).
Integrating radiomic features with clinical and dosimetric ones improves the prediction of RP, providing a more comprehensive tool for risk stratification in lung cancer patients undergoing radiotherapy.
This study showed that identifying high-risk patients for RP could allow for a more personalized treatment planning to reduce their risk, such as adjusting radiation dose constraints, introducing protective measures, or enhancing follow-up care.
Explore this infographic to learn more about this project and the predictive model developed by Evin et al’s.
Evin. C, et al. Clin Lung Cancer. 2024 Nov 20:S1525-7304(24)00248-1. doi: 10.1016/j.cllc.2024.11.003
Principal Investigator: Eleonor Rivin del Campo, MD, PhD
SOPHiA DDM™ for Radiomics and SOPHiA DDM™ for Multimodal are concepts in development. May not be available for sale.
SOPHiA GENETICS products are for Research Use Only and not for use in diagnostic procedures unless specified otherwise.
SOPHiA DDM™ Dx Hereditary Cancer Solution, SOPHiA DDM™ Dx RNAtarget Oncology Solution and SOPHiA DDM™ Dx Homologous Recombination Deficiency Solution are available as CE-IVD products for In Vitro Diagnostic Use in the European Economic Area (EEA), the United Kingdom and Switzerland. SOPHiA DDM™ Dx Myeloid Solution and SOPHiA DDM™ Dx Solid Tumor Solution are available as CE-IVD products for In Vitro Diagnostic Use in the EEA, the United Kingdom, Switzerland, and Israel. Information about products that may or may not be available in different countries and if applicable, may or may not have received approval or market clearance by a governmental regulatory body for different indications for use. Please contact us to obtain the appropriate product information for your country of residence.
All third-party trademarks listed by SOPHiA GENETICS remain the property of their respective owners. Unless specifically identified as such, SOPHiA GENETICS’ use of third-party trademarks does not indicate any relationship, sponsorship, or endorsement between SOPHiA GENETICS and the owners of these trademarks. Any references by SOPHiA GENETICS to third-party trademarks is to identify the corresponding third-party goods and/or services and shall be considered nominative fair use under the trademark law.