Adaptivesplit

Adaptivesplit optimizes allocation of samples between model discovery and external validation to maximize predictive performance while preserving validation power in translational biomedical studies.

Key Features:

• Adaptive Sample Splitting: Implements adaptive sample splitting to allocate data between training (discovery) and external validation to maximize predictive performance without compromising validation power.
• Optimization of Validation Efforts: Determines the optimal point to cease model discovery based on a predefined "sample size budget" to ensure external validation has sufficient power.
• Public Disclosure for Credibility: Advocates separation of discovery and validation through public disclosure of feature processing steps and model weights to enhance transparency and credibility.
• Implementation as Python Package: Provided as a Python package for integration into computational workflows.
• Empirical Validation: Validated using datasets comprising over 3,000 participants across four different studies to demonstrate effectiveness in diverse research contexts.

Scientific Applications:

• Translational Medicine: Balances model discovery and external validation in translational medical studies to reduce overfitting and improve replicability.
• Genomics: Mitigates replicability issues and effect size inflation in genomic predictive modeling.
• Proteomics: Supports robust external validation of proteomic predictive models.
• Personalized Medicine: Preserves validation power for development of predictive models used in personalized medicine.

Methodology:

Adaptive sample splitting that optimizes allocation between discovery and external validation based on a predefined sample size budget, and public disclosure of feature processing steps and model weights; implemented as a Python package.