pipeComp

pipeComp provides a flexible R framework for comparing computational pipelines by handling interactions between analysis steps and relying on multi-level evaluation metrics.

Key Features:

• Pipeline representation: The pipelineDefinition class represents pipelines as a set of functions consecutively executed on the output of the previous one, optionally accompanied by step-wise evaluation and aggregation functions.
• Efficient execution: The runPipeline function proceeds through all combinations of arguments, avoiding recomputing the same step twice and compiling evaluations on the fly to avoid storing potentially large intermediate data.
• Extensibility: The framework can integrate any other step, tool, or evaluation metric, allowing extensible benchmarks and applications to other fields.
• Multi-level evaluation: The framework relies on multi-level evaluation metrics for assessing pipeline performance.

Scientific Applications:

• Single-cell RNA-sequencing pipeline benchmarking: Benchmarking of single-cell RNA-sequencing analysis pipelines using simulated and real datasets with known cell identities, covering common methods of filtering, doublet detection, normalization, feature selection, denoising, dimensionality reduction, and clustering.
• Differential expression analysis: Study of the impact of removal of unwanted variation on differential expression analysis.

Methodology:

The framework uses a pipelineDefinition class to define pipelines as consecutive functions with optional step-wise evaluation and aggregation functions, and a runPipeline function to execute all parameter combinations efficiently with caching and on-the-fly evaluation compilation.