HTScodecs

HTScodecs implements enhanced compression of high-throughput sequencing (HTS) data by applying CRAM 3.1 improvements to reduce the storage footprint of CRAM and BAM sequencing files.

Key Features:

• Enhanced Compression Efficiency: CRAM 3.1 in HTScodecs achieves ~7–15% smaller files than CRAM 3.0 for Illumina data and ~50–70% smaller files relative to BAM for the datasets reported.
• Support for Modern Sequencing Technologies: Updates accommodate short-read (Illumina) and long-read sequencing data, with larger compression gains for Illumina and more modest gains for long-read data due to higher-entropy signals.
• Algorithmic Enhancements: Implements advanced compression algorithms and practical optimizations tailored to the specific characteristics of sequencing data, including handling of high-entropy signals.
• Integration with HTSlib and CRAM/BAM-based Pipelines: CRAM 3.1 enhancements are implemented to be compatible with HTSlib and existing CRAM/BAM workflows.

Scientific Applications:

• Storage and Transfer of Sequencing Data: Reduces file sizes to facilitate more efficient storage and transfer of HTS datasets without compromising data integrity.
• Large-scale Genomic Projects: Enables more efficient management of data in population studies and other extensive sequencing efforts.
• Clinical and Translational Genomics: Supports storage and sharing needs in personalized medicine workflows by reducing sequencing data footprint.
• Evolutionary Biology: Facilitates handling and exchange of large sequencing datasets used in evolutionary and comparative genomics studies.

Methodology:

Implements CRAM 3.1-specific updates and advanced compression algorithms with algorithmic enhancements and practical optimizations tailored to sequencing data characteristics, including methods to address high-entropy signals in long-read sequencing.