MetaCacheSpark

MetaCacheSpark performs alignment-free k-mer–based metagenomic classification and quantification of species from whole genome shotgun sequencing data, extending the All-Food-Sequencing (AFS) methodology to detect and quantify animal, plant, and microbial components.

Key Features:

• Alignment-free k-mer methodology: Employs an alignment-free k-mer–based approach to classify and quantify species from sequence reads and is orders-of-magnitude faster than alignment-based AFS pipelines.
• Performance metrics: Reports lower false-positive rates and higher quantification accuracy compared to CLARK, Kraken2, and Kraken2+Bracken.
• Database partitioning: Implements an efficient database partitioning scheme to minimize memory requirements for extensive reference genome collections and supports AFS-MetaCache on workstations and MetaCacheSpark on Spark clusters.
• Scalability: Scales to large collections of complex eukaryotic and bacterial reference genomes for big-data applications.

Scientific Applications:

• Biosurveillance and food testing: Provides fast, sequence-based screening for whole genome shotgun sequencing–based biosurveillance and comprehensive food testing to detect diverse biological components in complex samples.
• Broad-scale metagenomic screening: Enables broad-scale metagenomic screening across research domains due to its scalability and efficiency.

Methodology:

Uses an alignment-free k-mer based analysis of sequence reads combined with database partitioning to reduce memory usage, enabling operation on workstations (AFS-MetaCache) or distributed Spark-based compute clusters (MetaCacheSpark).