minicore

minicore performs scalable clustering of large-scale single-cell RNA-seq (scRNA-seq) datasets by combining vectorized weighted reservoir sampling with k-means variants to identify cell groups and evaluate dissimilarity measures.

Key Features:

• Efficient Center Finding: Uses a vectorized weighted reservoir sampling algorithm to accelerate selection of initial k-means++ centers, enabling processing of datasets containing 4 million cells in 1.5 minutes using 20 threads.
• Versatile Distance Measures: Supports Euclidean distance, Jensen-Shannon Divergence, Kullback-Leibler Divergence, and Bhattacharyya distance, which are advantageous for count data and probability distributions from scRNA-seq.
• Memory Efficiency: Optimized to perform clustering on datasets comprising millions of cells using less than 10 GiB of RAM.
• Performance Optimization: Demonstrates lower-cost centering compared to scikit-learn and shows minimal speed differences (<2-fold) across distance measures when priors are carefully managed.
• Comprehensive Clustering Pipeline: Integrates k-means++, local search++, and minibatch k-means to enable rapid clustering of high-throughput scRNA-seq datasets within minutes.

Scientific Applications:

• Sparse and Dense Data Handling: Handles sparse count data directly from typical scRNA-seq experiments and dense data after dimensionality reduction.
• Cell Group Definition: Defines cell groups with similar expression profiles for downstream interpretation.
• Distance Measure Evaluation: Provides insights into which dissimilarity measures yield clusterings most consistent with known cell type labels.

Methodology:

Initial centers are selected via vectorized weighted reservoir sampling for k-means++; clustering proceeds with k-means++, local search++, and minibatch k-means and supports Euclidean, Jensen-Shannon, Kullback-Leibler, and Bhattacharyya distances with optional priors during centering.