HoneyBee

HoneyBee integrates multimodal biomedical data for oncology applications by processing clinical data (structured and unstructured), whole-slide images, radiology scans, and molecular profiles to generate unified patient-level embeddings using domain-specific foundation models and fusion strategies.

Key Features:

• Multimodal Data Integration: Processes clinical data (structured and unstructured), whole-slide images, radiology scans, and molecular profiles.
• Foundation Model-Driven Embeddings: Generates unified patient-level embeddings using domain-specific foundation models and fusion strategies.
• Clinical Embedding Performance: Achieves 98.5% classification accuracy and 96.4% precision@10 in patient retrieval as the strongest single-modality performer.
• Survival Prediction: Achieves the highest survival prediction concordance indices across most cancer types, with multimodal fusion providing complementary benefits for specific cancers.
• Large Language Model Evaluation: Comparative evaluation of four large language models including Qwen3 and specialized medical models for clinical text representation.

Scientific Applications:

• Survival Prediction: Predicts patient survival outcomes with multimodal fusion improving overall survival prediction beyond clinical features alone.
• Cancer-Type Classification: Classifies cancer types with 98.5% accuracy using clinical embeddings.
• Patient Similarity Retrieval: Retrieves similar patients with 96.4% precision@10 using clinical embeddings.
• Cohort Clustering: Clusters patient cohorts based on unified multimodal embeddings.

Methodology:

HoneyBee uses domain-specific foundation models and fusion strategies to generate unified patient-level embeddings from multimodal data, evaluated on 11,400+ patients across 33 cancer types from The Cancer Genome Atlas (TCGA), with task-specific fine-tuning for heterogeneous data such as pathology reports.