Umpire 2.0

Umpire 2.0 simulates complex, heterogeneous, and mixed-type clinical and molecular datasets with known subgroup identities, dichotomous outcomes, and time-to-event (survival) data for benchmarking supervised and unsupervised machine learning methods.

Key Features:

• Simulation of mixed-type data: Generates correlated and heterogeneous binary, continuous, categorical, or mixed-type variables to reflect complex clinical and molecular measurements.
• Known ground truth generation: Produces explicit subgroup identities and survival parameters to provide ground truth for method evaluation.
• Dichotomous and time-to-event outcomes: Supports simulation of dichotomous outcomes and time-to-event (survival) data.
• Additive noise application: Applies configurable additive noise to simulate measurement error and variability in data quality.
• Discretization capabilities: Converts continuous data into single or mixed discrete formats as required.
• Scalability across sample sizes and feature spaces: Handles datasets ranging from small clinical trials to large electronic health record cohorts comprising thousands of patients.
• Support for ML evaluation: Produces datasets tailored for evaluating both supervised and unsupervised machine learning methods.

Scientific Applications:

• Benchmarking machine learning algorithms: Enables controlled evaluation of supervised and unsupervised methods using datasets with known ground truth.
• Method development and validation: Facilitates testing algorithm robustness to heterogeneity, correlation structures, noise, and different outcome types.
• Clinical machine learning research: Provides clinically realistic simulated datasets for assessing prediction and classification approaches in healthcare contexts.

Methodology:

Simulation is implemented as a series of user-defined modules with adjustable parameters to generate correlated heterogeneous variables, survival parameters, known subgroups, additive noise, and discretized outputs for tailored simulation scenarios.