CIndex

CIndex provides space-efficient compressed indexing and fast query retrieval for large FASTQ files produced by ultrahigh-throughput next-generation sequencing (NGS) instruments.

Key Features:

• Compressed Indexing: Uses the Burrows-Wheeler transform (BWT) combined with wavelet trees, hybrid encoding techniques, and succinct data structures including REF and Rγ tables to build a compressed index for FASTQ files.
• Space Efficiency: Demonstrates reduced storage requirements, using 2.7–41.66% less space for count/locate/extract queries on reads and 2.86–14.88% less space for extracting records in FASTQ files.
• Fast Retrieval: Achieves substantial speedups for query operations, with count queries 70–167.16× faster, locate queries 1.44–35.57× faster, extract-reads queries 1.3–55.4× faster, and FASTQ record extraction 3.13–20.1× faster compared to baseline solutions.
• Adaptability: Can be adapted as a general-purpose text index for datasets beyond FASTQ while retaining compressed storage and fast query capabilities.

Scientific Applications:

• Genomic data management: Enables space-efficient storage and rapid retrieval of large-scale sequencing outputs from NGS instruments to support downstream analyses.
• Variant calling: Facilitates fast extraction and counting of reads required for variant detection workflows.
• Genome assembly: Supports efficient access to read sets needed during assembly processes.
• Metagenomics: Provides compressed indexing and fast retrieval for large metagenomic sequencing datasets used in taxonomic and functional profiling.

Methodology:

Implements the Burrows-Wheeler transform (BWT), wavelet trees, hybrid encoding, and succinct data structures (including REF and Rγ tables) to compress and index FASTQ data.