modelBuildR

modelBuildR facilitates model building and feature selection in omics-based biomedical research to identify features associated with true biological groups despite error-prone human or expert classifications.

Key Features:

• Error-Prone Classification Handling: Incorporates methods to mitigate the impact of discordance rates (5–15%) in human classifications on model accuracy.
• Feature Pre-filtering Heuristic (V1): Implements a heuristic that applies univariate testing, multiplicity adjustment, and cross-validation with switched dependent and independent variables for feature preselection.
• Model Training and Evaluation: Trains logistic regression or linear models with backward selection based on Akaike Information Criterion (AIC) and evaluates performance using Receiver Operating Characteristic (ROC) curves and multiclass-ROC metrics against ground truth.
• Comparative Benchmarking: Provides a framework to benchmark multiple feature selection and classification methods to assess their effectiveness at retrieving true biological-group-associated features.
• Simulation-based Assessment: Supports simulation of datasets with up to 10,000 features and varying sample sizes to evaluate method performance under controlled ground truth and classification error models.

Scientific Applications:

• Glioblastoma methylation analysis: Applied to TCGA-GBM 450k methylation array data to identify prognostically different G-CIMP negative glioblastoma tumors and demonstrate prognostic separation with differences in overall survival.
• Omics model building and classification transfer: Used in omics research to retrieve features associated with true groups and support transfer of classifications across datasets despite rating errors.

Methodology:

Simulates datasets with up to 10,000 features and varying sample sizes with a subset associated to ground truth groups; evaluates binary and semi-quantitative ratings with different error probabilities; applies the V1 pre-filtering heuristic (univariate testing, multiplicity adjustment, cross-validation with switched variables); trains logistic or linear models via backward selection by AIC; and compares model performance to ground truth using ROC and multiclass-ROC metrics, reporting median AUC ranks versus standard methods (V2).