DrEval

DrEval evaluates cancer drug response models through unbiased, statistically sound benchmarking with baseline comparisons, ablation studies, and cross-study validation to improve reliability in personalized medicine.

Key Features:

• Baseline Models: Provides baseline and literature models with consistent hyperparameter tuning, including a naive model predicting mean drug and cell line effects and properly tuned tree-based ensemble baselines.
• Statistical Evaluations: Implements statistically sound evaluations to address pseudoreplication, biased evaluation metrics, and overly optimistic performance estimates.
• Ablation Studies: Enables systematic ablation studies to assess model component contributions.
• Cross-Study Benchmarks: Supports cross-study benchmarks to evaluate generalization and reduce data leakage.
• Visualizations: Generates publication-ready visualizations for result communication.

Scientific Applications:

• Drug Response Prediction Evaluation: Benchmarks deep learning and other complex models against simpler baselines using large-scale drug sensitivity screen data from cancer cell line omics profiles.
• Reproducibility Assessment: Identifies and mitigates non-reproducible models, data leakage, and inconsistent viability data that reduce comparability across models.
• Clinical Translation: Provides a standardized benchmark to ensure clinically relevant and robust assessment of cancer drug response prediction models.

Methodology:

DrEval is implemented as a Python package (drevalpy) and an nf-core pipeline (drugresponseeval), with preprocessing scripts for drug response data preparation.