Fast-HBR

Fast-HBR removes duplicate reads from Next-Generation Sequencing (NGS) data to improve the accuracy and efficiency of downstream analyses.

Key Features:

• Memory Efficiency: Represents reads as integer hash values stored in hash tables instead of full string sequences to reduce memory usage.
• Speed and Performance: Processes input on-the-fly from FASTQ or FASTA files and avoids storing full string representations to enable faster duplicate detection.
• De-novo Duplicate Removal: Identifies and removes duplicates without requiring a reference genome.
• Direct Output Writing: Writes unique reads directly to an output file during processing to minimize intermediate storage and I/O.
• Implementation: Implemented in Python 3.

Scientific Applications:

• Environmental genomics: Removes duplicate reads from NGS datasets derived from environmental samples where reference genomes may be absent.
• Species identification and biodiversity assessment: Reduces duplicate-induced biases in read counts to improve accuracy of species identification and biodiversity estimates.
• Ecological studies: Enhances reliability of community and ecological analyses by eliminating redundant reads prior to downstream processing.

Methodology:

Represents reads as integer hash values in hash tables, processes FASTQ/FASTA input on-the-fly to perform de-novo duplicate detection and removal, and writes unique reads directly to an output file; implemented in Python 3.