MultimodalPrognosis

MultimodalPrognosis is a software tool that predicts cancer patient survival using multimodal data, including clinical information, mRNA expression, microRNA expression, and histopathology whole slide images (WSIs). The tool employs a multimodal neural network-based model to estimate prognosis for 20 cancer types.

Key features of MultimodalPrognosis include:

1. Unsupervised encoder: The tool compresses the four data modalities into a single feature vector for each patient, enabling efficient multimodal data handling.

2. Resilient multimodal dropout method: This approach allows the model to handle missing data effectively.

3. Tailored encoding methods: Deep highway networks extract features from clinical and genomic data, while convolutional neural networks extract features from WSIs.

4. Pancancer and single cancer survival prediction: The model is trained on pan-cancer data and can predict overall survival for both single cancer sites and pan-cancer cases, achieving a C-index of 0.78.

5. Flexible and informative patient data representation: The tool efficiently analyzes WSIs and represents patient multimodal data unsupervised and informatively.