scRNA-seq

scRNA-seq assigns cell type labels to clusters derived from single-cell RNA-sequencing (scRNA-seq) data to benchmark and compare labeling methods and improve reproducibility of cell-type annotation.

Key Features:

• Benchmarked Methods: Benchmarks five methods: ORA (over-representation analysis), GSEA (Gene Set Enrichment Analysis), GSVA (Gene Set Variation Analysis), CIBERSORT, and METANEIGHBOR.
• ORA: Performs traditional over-representation analysis on cluster marker gene sets.
• GSEA: Evaluates gene set enrichment across the entire ranked list of genes.
• GSVA: Estimates variation of pathway activity across samples in an unsupervised manner.
• CIBERSORT: Applies deconvolution algorithms to estimate cell type proportions from bulk tissue gene expression data.
• METANEIGHBOR: Uses network-based neighbor voting for label assignment.
• Evaluation Metrics: Evaluates methods using receiver operating characteristic (ROC) curve analysis and precision-recall analysis.
• Performance Summary: Reported average ROC AUC 0.91 ± 0.06 and average precision-recall AUC 0.53 ± 0.24 across methods and datasets.
• Robustness Testing: Assesses robustness via cell type signature subsampling simulations, with GSVA showing particular robustness.
• Speed and Sensitivity: Notes METANEIGHBOR and GSVA are fast, while CIBERSORT and METANEIGHBOR show sensitivity when analyses are restricted to expected cell types.
• Signature Size Effect: Finds the number of genes in cell type signatures influences performance, with smaller signatures more prone to incorrect results.
• Test Datasets and Platforms: Tested on five scRNA-seq datasets including human liver, Tabula Muris (11 mouse tissues), two human PBMC datasets, and mouse retinal neurons using Drop-seq, 10X Chromium, and Seq-Well with ~3,700–68,000 cells.

Scientific Applications:

• Cell-type Annotation: Assigns and benchmarks cell type labels for scRNA-seq clusters in human and mouse tissues.
• Cross-platform Benchmarking: Compares labeling method performance across Drop-seq, 10X Chromium, and Seq-Well datasets.
• Method Evaluation: Quantifies method performance using ROC and precision-recall metrics to identify variability across datasets and approaches.
• Signature Robustness Assessment: Evaluates the impact of cell type signature composition and subsampling on annotation accuracy.

Methodology:

Benchmarks ORA, GSEA, GSVA, CIBERSORT, and METANEIGHBOR on scRNA-seq cluster gene signatures, evaluates performance with receiver operating characteristic and precision-recall analyses, and performs cell type signature subsampling simulations.