scRNAseq_Benchmark

scRNAseq_Benchmark evaluates automatic cell identification methods for single-cell RNA sequencing (scRNA-seq) data to compare classifier performance across diverse datasets.

Key Features:

• Classifier set: Evaluates 22 classification methods encompassing single-cell-specific classifiers and general-purpose classifiers.
• Datasets: Uses 27 publicly available scRNA-seq datasets that vary in size, sequencing technology, species, and complexity.
• Experimental setups: Implements intra-dataset (within-dataset) and inter-dataset (across-dataset) prediction experiments.
• Performance metrics: Reports accuracy, percentage of unclassified cells, and computation time.
• Sensitivity analyses: Assesses classifier sensitivity to input features, cell population sizes, annotation levels, and dataset diversity.
• Top-performing classifier: Notes that the general-purpose support vector machine (SVM) classifier demonstrates superior performance across examined conditions.
• Workflow automation: Includes a Snakemake workflow to automate benchmarking analyses and integration of classifiers and datasets.

Scientific Applications:

• Classifier evaluation: Compare and evaluate automatic cell identification methods for scRNA-seq studies.
• Method selection: Inform selection of classifiers for experiments with varying dataset complexity, annotation depth, and cross-dataset transfer.
• Impact assessment: Determine how input feature choice, cell population size, annotation level, and dataset diversity affect classification outcomes.

Methodology:

Systematic evaluation of 22 classifiers on 27 scRNA-seq datasets using intra-dataset and inter-dataset prediction experiments, with performance measured by accuracy, percentage of unclassified cells, and computation time, plus sensitivity analyses to input features, cell population sizes, annotation levels, and dataset diversity; benchmarking analyses are automated with a Snakemake workflow.