gdsfmt

gdsfmt provides a high-level interface to CoreArray Genomic Data Structure (GDS) files for storing and accessing large-scale array-oriented genomic data, enabling efficient handling and analysis of whole-genome sequencing (WGS) datasets that exceed available RAM.

Key Features:

• CoreArray GDS interface: Exposes a high-level programmatic interface to CoreArray Genomic Data Structure (GDS) files for hierarchical, array-oriented data and associated metadata.
• Portability and scalability: GDS files support cross-platform use and a hierarchical structure for efficient organization and retrieval of large genomic datasets.
• Efficient data storage: Multiple compression options reduce file sizes compared with text-based VCF; benchmarks with 1000 Genomes Phase 3 show reduction from 14.0 Gb (VCF) to 2.6 Gb using SeqArray.
• High-performance data access: Optimized for parallel computing and demonstrates genotype reading speeds two to three times faster than the htslib C library when handling BCF files.
• Rapid computational operations: Supports fast computations such as allele frequency calculations, outperforming PLINK v1.9 and vcftools by over 16× when operating on VCF or BCF data.
• Optimized for large-scale analysis: Designed to manage datasets much larger than available RAM, enabling analyses involving numerous genetic variants and complex metadata.

Scientific Applications:

• WGS variant genotype storage and analysis: Persistent storage and retrieval of variant genotypes from whole-genome sequencing datasets.
• Large-scale population genetics computations: Scalable allele frequency and genotype-based analyses on population-scale datasets.
• Integration with R/Bioconductor: Use within the R/Bioconductor ecosystem to enable downstream genomic analyses and interoperability with other Bioconductor packages.

Methodology:

Implements an array-oriented CoreArray GDS format that preserves VCF functionality, employs multiple compression options to reduce file size, and is optimized for parallel computing to enable rapid access and manipulation of genomic data.