streammd

streammd performs fast, memory-efficient, single-pass duplicate marking for bulk sequence analysis using a Bloom filter to enable scalable preprocessing for downstream analyses such as variant calling and expression quantification.

Key Features:

• Speed and Efficiency: Performs rapid duplicate detection with high accuracy.
• Low Memory Usage: Requires substantially less memory compared to tools like SAMBLASTER, enabling large-scale sequencing analyses.
• Single-Pass Operation: Processes sequencing data in a single pass, reducing computational overhead.
• Bloom filter-based Marking: Uses a Bloom filter approach for memory-efficient identification of duplicate templates.
• Output Fidelity: Closely reproduces outputs from Picard MarkDuplicates.
• Implementation: Implemented in C++.

Scientific Applications:

• Duplicate Marking in Preprocessing: Identifies and marks duplicate templates during preprocessing of bulk sequence data.
• Variant Calling: Facilitates downstream variant calling by removing or flagging duplicate reads that can bias variant detection.
• Expression Quantification: Supports accurate expression quantification by accounting for duplicate templates in sequencing libraries.
• Genomic Studies and Personalized Medicine: Scales duplicate marking for large libraries used in genomic studies and personalized medicine research.

Methodology:

Leverages a Bloom filter-based, single-pass duplicate marking approach and closely reproduces outputs from Picard MarkDuplicates.