Tabular Data Curator

Tabular Data Curator automates curation and quality assessment of tabular clinical datasets to detect duplicated fields, lexical similarities, high correlations, outliers, inconsistencies, and missing values, and to standardize and harmonize data for cross-cohort analyses.

Key Features:

• Automated Data Curation: Performs automated identification and resolution of duplicated fields, outliers, inconsistencies, and missing values in tabular datasets.
• Detection and Tracking: Employs algorithms to detect lexical similarities and high correlations among features and to track inconsistencies and outliers within datasets.
• Handling Missing Values: Applies data imputers to manage missing values and preserve dataset completeness.
• Data Standardization and Harmonization: Standardizes formats and terminologies and harmonizes datasets from different sources to enable consistent cross-cohort analyses.

Scientific Applications:

• Primary Sjögren's Syndrome (pSS) case study: Matched over 85% of pSS-related medical terms across two clinical cohorts.
• Clinical cohort harmonization: Enables integration and comparison of clinical datasets across cohorts by aligning measures and terminologies.
• Biomedical research data preparation: Provides curated, standardized input data for downstream biomedical analyses and analytics services.

Methodology:

The architecture comprises a Detection Layer that identifies inconsistencies, missing values, outliers, and similarities; a Standardization Layer that applies standardization techniques to enforce uniform formats and terminologies; and a Harmonization Layer that aligns datasets with standardized measures for integration.