spotless-benchmark

spotless-benchmark provides a Nextflow pipeline for benchmarking cell type deconvolution methods in spatial transcriptomics data through a simulation engine and standardized evaluation framework.

Key Features:

• Simulation Engine: synthspot generates silver standards from single-cell RNA-sequencing data for benchmarking deconvolution methods.
• Standardized Evaluation Framework: The pipeline enables assessment and comparison of deconvolution tools based on performance, stability across different reference datasets, and scalability.
• Reproducible Benchmarking: Results are reproducible in a Nextflow pipeline that allows users to generate synthetic data, run deconvolution methods, and optionally benchmark them on their own datasets.

Scientific Applications:

• Cell Type Deconvolution Benchmarking: The pipeline was used to benchmark 11 deconvolution methods using 63 silver standards, 3 gold standards, and 2 case studies on liver and melanoma tissues.
• Performance Evaluation Under Challenging Conditions: The framework evaluates method performance in datasets with highly abundant or rare cell types, revealing significant performance decreases across all methods in these scenarios.

Methodology:

Silver standards are generated from single-cell RNA-sequencing data using the synthspot simulation engine, while gold standards are generated by pooling single cells from targeted spatial transcriptomics data. Methods are evaluated based on performance, stability across different reference datasets, and scalability. The entire workflow is implemented as a Nextflow pipeline.