cd-hit

cd-hit clusters nucleotide and protein sequences to reduce redundancy and produce representative sequence sets for downstream genomic and proteomic analyses.

Key Features:

• Redundancy reduction: Clusters sequences into representative groups based on sequence identity thresholds to reduce redundancy in large datasets.
• Nucleotide and protein support: Handles both nucleotide (DNA/RNA) and protein sequence databases.
• Specialized programs: Includes cd-hit-2d for comparing two protein datasets, cd-hit-est for clustering DNA/RNA databases, and cd-hit-1d for nucleotide dataset comparison.
• Parallelization: Implements parallelization strategies to accelerate processing of large sequence collections.
• Short-word filters: Uses short-word filters and techniques to improve their use, enabling substantial speed increases (reported up to ~100-fold).
• High performance: Optimized for ultrafast clustering with reported orders-of-magnitude speedups relative to BLAST and example performance clustering ~560,000 protein sequences in about two hours on a high-end PC.
• Sequence identity thresholds: Supports clustering at different sequence identity levels to control cluster granularity.

Scientific Applications:

• Dereplication for downstream analysis: Reduces dataset size and computational load for downstream genomic and proteomic analyses by producing representative sequence sets.
• Public database clustering: Applied to cluster and compress public protein databases such as NCBI NR, SwissProt, and PDB.
• Comparative dataset analysis: Compares two protein datasets to identify shared and unique sequences (cd-hit-2d).
• DNA/RNA clustering: Clusters nucleotide (DNA/RNA) sequence databases for applications in transcriptomics and metagenomics (cd-hit-est, cd-hit-1d).
• NGS-scale processing: Scales to very large sequence collections produced by next-generation sequencing (NGS) technologies.

Methodology:

Clusters sequences into representative groups using sequence identity thresholds, employs short-word filters and techniques to optimize filter use, and applies parallelization strategies to accelerate processing.