Context & Overview

Multimodal Artificial Intelligence approaches combine diverse data modalities, such as cancer multiomics (e.g., genomics, proteomics, metabolomics, radiomics), imaging, histopathology, clinical and biological data, and other real-world data, into unified analytical models (Figure 1).

Unlike traditional single-biomarker approaches, MMAI captures relationships across biological and clinical scales, linking molecular changes to patient outcomes. This holistic approach enhances predictive accuracy, interpretability, and clinical utility, offering an unprecedented potential to transform every phase of cancer research and management.

By bridging the gap between biological complexity and computational capability, MMAI unlocks richer insights that can transform how prevention, diagnosis, treatment, and drug development are conducted, with implications for health economics, regulatory policy, and global health equity (Figure 2).

Applications Across the Oncology Ecosystem

1. Care Pathway Optimization: Prevention and Early Detection

MMAI is emerging as a powerful tool for personalized prevention, and population-level screening by integrating clinical data to predict cancer risk and guide early interventions.

• Personalized prevention strategies: Epidemiological MMAI models can analyze population health data to predict disease risk factors and recommend tailored interventions, from patient monitoring to prevention strategies.
• Screening and risk stratification: The potential for MMAI predictive models to enable targeted screening and risk stratification has been shown in several studies, with some of these algorithms already outperforming traditional screening approaches. The Sybil AI system for lung cancer risk prediction and the MONAI initiative are two examples that show how MMAI approaches allow for earlier interventions across the cancer care pathway that may improve patient outcomes.
• AI-enhanced early detection: The integration of deep learning tools into clinical workflows improve the detection sensitivity for subtle tumor lesions and changes. For example, emerging AI-assisted liquid biopsy models are advancing non-invasive detection of circulating tumor DNA (ctDNA), expanding early intervention opportunities.

2. Diagnosis and Prognosis

MMAI enables a shift from reactive to data-informed decision-making by synthetizing diverse data types into meaningful insights.

• AI-assisted diagnosis: Numerous AI-assisted diagnostic approaches have demonstrated diagnostic accuracy comparable to expert pathologists with high sensitivity and specificity. Among these, the authors highlight 1) lightweight architectures, such as ShuffleNet to infer genomic alterations from histology slides, reducing sequencing time and cost; 2) AI-powered imaging systems to enhance tumor detection, lesion characterization, and disease staging. The application of these tools improves prognostic accuracy and survival prediction.
• Prognosis and outcome prediction: MMAI models have proven efficient to forecast progression and therapy response. Models such as Pathomic Fusion and Stanford’s MUSK demonstrated superior performance in predicting cancer progression and therapy response compared to existing single-modality approaches. AstraZeneca’s ABACO platform and TRIDENT machine learning (ML) multimodal model, exemplify MMAI in action, by integrating multimodal data to identify predictive biomarkers for targeted treatment selection, optimize therapy response predictions, and improve patient stratification.

3. Personalized Treatment and Patient Management

Precision oncology relies on accurately identifying meaningful patient subgroups. MMAI enables this by integrating hundreds of biological and clinical variables beyond the scope of traditional analytics.

• Treatment selection: MMAI models can integrate complex multimodal datasets to recommend therapy combinations. For instance, multimodal models used in the DREAM Challenge and TransNEO/ARTemis breast cancer studies consistently outperformed unimodal benchmarks in predicting treatment response.
• Remote and AI-assisted monitoring and management: Continuous data from telehealth and wearable sensors can feed MMAI models that monitor symptoms, detect early complications, and trigger clinical interventions. Through multimodal integration of remote patient monitoring and conventional data streams, AstraZeneca’s ABACO platform captured complementary information, allowing for the improvement of the predictive performance and potentially enabling oncologists to dynamically adjust treatment strategies based on near real-time feedback loops.
• Quality of life: Improving or maintaining patients’ quality of life past systemic treatments remains a challenge. The integration of electronic patient-reported outcomes (ePROs) into the multimodal workstream expands the MMAI paradigm beyond survival to patient-centered care.

4. Drug Development and Clinical Research

MMAI is revolutionizing the entire drug development process, from discovery to development and clinical validation. Through smarter patient recruitment, synthetic control arms, and adaptive trial design, MMAI accelerates drug discovery and optimizes clinical trials, ultimately streamlining approvals and reducing costs.

• AI-accelerated drug discovery and development: AI-driven platforms, like BenevolentAI, identify drug candidates faster by integrating multi-omics data and structure-activity relationships. It is now estimated that AI-designed molecules advance to clinical trials twice as fast as traditionally developed drugs.
• Clinical trial optimization: AI-driven strategies can lead to more efficient and precise drug trials based on real-world evidence (RWE) and comparator cohorts (“digital twin”), therefore reducing the dependence on traditional control groups, particularly valuable in rare cancers or small patient populations. By providing eligibility-matching engines, real-time adaptive randomization, and synthetic control arms, MMAI approaches further streamline the clinical trial process, reduce manual screening time and costs, fasten regulatory submissions and improve decision-making in drug development.
• Patient recruitment enhancement: Multimodal models streamline patient recruitment by matching individuals to biomarker-driven studies, reducing screening burden and improving trial representativeness. Algorithms such as Trial Pathfinder suggest that up to 17% more patients could qualify for oncology trials with optimized eligibility criteria.

Health System Value and Benefits

It’s undeniable that MMAI brings measurable advantages for health systems struggling with the fast pace and rising oncology costs, by improving patient-therapy matching, reducing overtreatment, and shortening diagnostic times.

Ways in which MMAI can enhance patient outcomes and cost-effectiveness

Interoperable data results in system-level gains	Improved diagnostic capability reduces overtreatment	Diagnostics and targeted treatment shift to increased cost-effectiveness
Deploying MMAI on harmonized datasets (e.g., FHIR and OMOP frameworks) allows for multiple types of data to be analyzed together rather than in silos, reducing human burden and increasing productivity.	Combining digital pathology, multiomics and other clinical variables, MMAI can uncover biomarker signatures that identify optimal treatment responders, ensuring patients receive the right treatment at the right time.	Leveraging AI-driven approaches allows for the reallocation of resources from low-value blanket treatment to high-value targeted intervention to optimize patients’ treatment plans.

Beyond ROI and productivity gains, MMAI models can further support health systems, especially in underserved regions of the world, such as Africa and Asia, where AI-assisted telepathology and teleradiology networks are already bridging diagnostic gaps, reinforcing MMAI’s role in advancing global cancer equity.

Overall, MMAI enhances cost-effectiveness, fosters data-driven reimbursement models, and promotes equitable, sustainable oncology care worldwide.

Challenges and Ethical Considerations

MMAI can materially improve cancer care, but despite all the progress in the development of MMAI-driven approaches, its adoption is challenged by fragmentation, trust and bias concerns, immature evaluation standards, and equity risks. Amongst the key aspects for ensuring a successful deployment, the authors highlight:

• Interoperability and standardization of data infrastructure.
• Transparent, explainable, and regulatory-compliant AI to ensure clinician trust and regulatory approval.
• Ethical governance and privacy protection, including federated- and swarm-learning frameworks to enable secure, distributed model training.
• Bias management, requiring preventive approaches and proactive design and validation in diverse populations to ensure generalizable outcomes.

Conclusion and Outlook

Multimodal AI stands at the forefront of an oncology revolution. By uniting biological, clinical, and digital data, MMAI provides an integrated lens through which cancer can be understood, predicted, and treated more effectively and timely.

Initiatives like ABACO  and TRIDENT demonstrate how MMAI can accelerate both real-world learning and clinical validation, shortening the path from data to discovery to patient benefit.

While challenges remain, the convergence of multimodal intelligence, precision medicine, and cross-industry collaboration marks a transformative step toward more predictive, equitable, and sustainable cancer care, at scale.

To learn more about SOPHiA DDM™ for Multimodal and our ongoing collaborations, explore our dedicated webpage.

Disclaimer
SOPHiA DDM™ for Multimodal is a concept in development. May not be available for sale.