MultimodalPrognosis

MultimodalPrognosis predicts patient survival across 20 cancer types by integrating clinical data, mRNA expression data, microRNA expression data, and histopathology whole slide images (WSIs) into multimodal prognostic models.

Key Features:

• Multimodal Data Integration: Integrates clinical data, mRNA expression data, microRNA expression data, and histopathology whole slide images (WSIs).
• Unsupervised Feature Encoding: Uses an unsupervised encoder to compress diverse data types into a single feature vector per patient.
• Missing-data Robustness: Employs a resilient multimodal dropout technique to handle missing data across modalities.
• Tailored Encoding Methods: Applies deep highway networks to clinical and genomic data and convolutional neural networks (CNNs) to WSIs for modality-specific feature extraction.
• Pancancer Training: Trained on pancancer data to predict both single-cancer and overall pancancer survival outcomes across 20 cancer types.
• Performance Metrics: Reports an overall concordance index (C-index) of 0.78.

Scientific Applications:

• Prognosis Prediction: Provides survival predictions across multiple cancer types to support prognostic biomarker research and inform personalized treatment strategies.

Methodology:

Unsupervised encoder compresses modalities into a single patient feature vector; resilient multimodal dropout handles missing data; deep highway networks encode clinical and genomic data; convolutional neural networks (CNNs) extract features from WSIs; model trained on pancancer data.