RabbitTClust

RabbitTClust clusters large-scale bacterial genome datasets using sketch-based distance estimation to provide rapid, memory-efficient genome clustering for comparative genomics and database-scale analyses.

Key Features:

• Speed and Scale: Clusters 113,674 complete bacterial genomes (455 GB FASTA) in under six minutes and 1,009,738 GenBank assembled bacterial genomes (4.0 TB FASTA) in approximately 34 minutes on a 128-core workstation.
• Memory Efficiency: Optimized for memory-efficient processing of terabyte-scale FASTA datasets.
• Parallelization and Streaming: Implements parallel processing and streaming to leverage modern multi-core systems for high-throughput computation.
• Dimensionality Reduction: Integrates dimensionality reduction methods to simplify large genomic datasets while preserving relevant distance relationships.
• Sketch-based Distance Estimation: Uses sketch-based distance estimation to approximate genomic distances efficiently.

Scientific Applications:

• Microbial Genome Clustering: Groups bacterial genomes to identify genetic similarities and differences among bacterial species.
• Database Curation and Redundancy Detection: Detects redundant genomes in repositories (e.g., 1,269 redundant genomes with identical nucleotide content found in RefSeq bacterial genomes) to inform curation and deduplication.
• Comparative Genomics, Diversity, Evolution and Taxonomy: Supports large-scale studies of microbial diversity, evolution, and taxonomy through scalable genome clustering.

Methodology:

Employs sketch-based distance estimation combined with dimensionality reduction and implements streaming and parallel processing on multi-core workstations.