FRESCO

FRESCO compresses referential biological sequence data by encoding differences to reduce storage and enable efficient handling of large genomic datasets.

Key Features:

• Referential Compression: Employs referential compression that stores only the differences between a target input sequence and a known reference sequence.
• Speed and Efficiency: Implements a compression algorithm reported to be one to two orders of magnitude faster than related methods while maintaining comparable compression ratios.
• Reference Sequence Optimization: Includes techniques for selecting and rewriting an optimal reference sequence to improve overall compressibility.
• Second-Order Compression: Applies referential compression iteratively to already-compressed files (second-order compression), achieving reported compression ratios exceeding 4,000:1 for human genomes.
• Evaluation on Large Datasets: Demonstrated on datasets comprising over 1,000 genomes (exceeding 3 TB) and a collection of Wikipedia page versions, showing capability for real-time compression on modern hardware.

Scientific Applications:

• Large-scale genomic storage: Reduces storage footprint for large collections of sequencing data, including human genomes.
• Variant and comparative genomics storage: Encodes differences against references to efficiently represent variants and multiple closely related sequences.
• Archival of versioned data: Applicable to archival compression of versioned documents and sequence revisions, as demonstrated on Wikipedia page versions.

Methodology:

Stores only differences between target sequences and reference sequences, selects and rewrites optimal references to improve compressibility, applies referential compression iteratively as second-order compression, and was evaluated on >1,000 genomes (>3 TB) and a collection of Wikipedia page versions.