julearn

julearn implements machine learning pipeline construction, validation, and evaluation in Python to enforce methodological rigor and reproducibility in research, with emphasis on correct cross-validation and prevention of information leakage.

Key Features:

• Modular pipeline components: Modular specification of preprocessing, feature extraction, model fitting, and evaluation steps.
• Explicit train/test separation: Explicit separation of train and test operations to ensure validity of generalization estimates.
• Cross-validation support: Built-in support for diverse cross-validation (CV) schemes and nested validation.
• Grouping and stratification: Configurable grouping and stratification within CV schemes.
• Fold-aware transformations: Automatic propagation of transformations within folds to prevent data leakage across splits.
• Bias prevention: Mechanisms to prevent common sources of information leakage and optimistic bias in evaluation.
• Implementation: Available as a Python library for constructing and evaluating ML pipelines.

Scientific Applications:

• MRI/EEG biomarker development: Supports rigorous evaluation workflows for MRI- and EEG-based biomarker studies.
• Neuroscience machine learning: Addresses misuse of cross-validation strategies and pipeline evaluation in neuroscience research.
• Reproducible ML research: Enables reproducible design and assessment of preprocessing, feature extraction, and model fitting pipelines on scientific datasets.

Methodology:

Modular specification of preprocessing, feature extraction, model fitting, and evaluation; explicit train/test separation; support for diverse CV schemes and nested validation; configurable grouping/stratification; automatic propagation of transformations within folds; and safeguards against information leakage and optimistic bias.