FASTAFS

FASTAFS virtualizes compressed FASTA archives to provide random-access decompression and metadata-aware integrity verification for DNA, RNA, and protein sequence data.

Key Features:

• File System Virtualization: Uses Filesystem in Userspace (FUSE) to expose compressed FASTA archives as virtual files while keeping metadata synchronized with data files.
• Random-Access Decompression: Employs bit encodings together with Zstandard (zstd) compression to enable fast random-access decompression of sequence data.
• Backward Compatibility: Maintains compatibility with traditional FASTA files to allow integration into existing bioinformatics workflows without requiring code changes.
• Metadata Accessibility: Exposes sequence-specific metadata such as nucleotide counts and checksums without requiring full-file decompression.
• Instance Tracking and Integrity Verification: Tracks FASTAFS instances system-wide and provides mechanisms for file integrity verification.
• Scriptable Access: Enables scripted access to virtualized sequence files for integration into computational pipelines.

Scientific Applications:

• Large-scale sequence analysis: Enables fast, random retrieval of sequences from compressed DNA, RNA, and protein FASTA archives for high-throughput analyses.
• Genomics: Supports genomic studies that require quick access to reference sequences and indexed regions from compressed archives.
• Transcriptomics: Facilitates transcriptomics workflows by allowing targeted retrieval of transcript sequences without full decompression.
• Proteomics and protein structure prediction: Supports proteomic databases and protein sequence retrieval for downstream structure prediction and analysis.
• Data management and integrity verification: Supports collaborative research and data provenance through exposed checksums and instance tracking.

Methodology:

Implements a FUSE-based virtual filesystem to expose compressed FASTA content; uses bit encodings with Zstandard (zstd) compression for random-access decompression; exposes nucleotide counts and checksums and tracks instances system-wide.