Benchmarker

Benchmarker evaluates gene and variant prioritization algorithms in genome-wide association studies (GWAS) to provide unbiased, data-driven comparisons based on per-SNP heritability.

Key Features:

• Unbiased benchmarking: Uses a leave-one-chromosome-out cross-validation framework combined with stratified linkage disequilibrium (LD) score regression to avoid reliance on "gold standard" genes.
• Per-SNP heritability metric: Quantifies algorithm performance by estimating per-SNP heritability attributable to prioritized genes or variants.
• Comparison across prioritization strategies: Evaluates methods based on annotated gene sets and gene expression data to compare their effectiveness in identifying likely causal genes for specific phenotypes.
• Statistical rigor against chance: Compares algorithm performance not only against each other but also against random expectation to assess significance.
• Multi-GWAS evaluation: Applied to 20 well-powered GWASs to assess consistency and robustness across diverse datasets.
• Direct method comparisons: Facilitates head-to-head comparisons of methods including DEPICT, MAGMA, and NetWAS, reporting that DEPICT and MAGMA outperform NetWAS in certain contexts.
• Integration assessment: Evaluates the impact of combining different data sources and algorithms on gene prioritization quality.

Scientific Applications:

• GWAS interpretation: Objectively assesses prioritization approaches to support mapping of GWAS associations to likely causal genes and variants.
• Follow-up study prioritization: Ranks candidate genes and variants for experimental validation based on estimated per-SNP heritability.
• Method selection: Guides selection of prioritization algorithms (e.g., DEPICT, MAGMA, NetWAS) and combinations of data sources for specific phenotypes.
• Functional mapping: Supports mapping genetic associations to functional implications using annotated gene sets and gene expression evidence.

Methodology:

Benchmarker implements leave-one-chromosome-out cross-validation, stratified LD score regression to estimate per-SNP heritability, comparisons against random expectation, direct comparisons among DEPICT, MAGMA, and NetWAS, and evaluations across 20 GWASs including assessments of combined data sources and algorithms.