DataSAIL

DataSAIL facilitates leakage-reduced data splitting for machine learning models in biological data applications by formulating the splitting problem as a combinatorial optimization problem.

Key Features:

• Combinatorial optimization formulation: DataSAIL formulates finding leakage-reduced data splits as a combinatorial optimization problem.
• NP-hardness proof: The problem of finding optimal leakage-reduced splits is proven to be NP-hard.
• Scalable heuristic: DataSAIL provides a scalable heuristic based on clustering and integer linear programming to solve the optimization problem.
• Versatile Python package: DataSAIL supports realistic evaluation of machine learning models for biological data intended for out-of-distribution scenarios.

Scientific Applications:

• Biomedical machine learning evaluation: DataSAIL enables realistic evaluation of biomedical machine learning models by preventing information leakage during data splitting.
• Out-of-distribution biological data scenarios: DataSAIL supports machine learning applications on biological data where models must generalize to out-of-distribution settings.

Methodology:

DataSAIL formulates leakage-reduced data splitting as a combinatorial optimization problem, solves it using a scalable heuristic based on clustering and integer linear programming, and empirically demonstrates impact on evaluating biomedical machine learning models.