ADS-HCSpark

ADS-HCSpark accelerates scalable variant calling in large-scale next-generation sequencing (NGS) datasets by parallelizing GATK HaplotypeCaller on Apache Spark to enable efficient and accurate variant detection from BAM files.

Key Features:

• Parallelization on Multi-Core and Multi-Node Systems: Implements parallelization of the GATK HaplotypeCaller algorithm to utilize multi-core processors and distributed Apache Spark clusters.
• Adaptive Data Segmentation Strategy: Employs dynamic adaptive data segmentation to mitigate computation skew and optimize workload distribution across resources.
• Overlapping Data Blocks for Enhanced Accuracy: Customizes the Hadoop-BAM library to partition BAM files into overlapped blocks, preserving adjacency information for accurate variant detection.
• High Scalability and Performance: Demonstrates improved throughput, reporting up to 74% faster than GATK3.8 HaplotypeCaller on single nodes, 57% faster than GATK4.0 HaplotypeCallerSpark, and 27% faster than SparkGA on multi-node setups.
• Accuracy: Maintains variant-calling accuracy exceeding 99% while accelerating processing of large NGS datasets.

Scientific Applications:

• Disease Research: Supports identification of genetic variants associated with diseases from large-scale sequencing data.
• Clinical Treatment and Diagnostics: Enables scalable variant calling for clinical genomics workflows and treatment-related genetic analyses.
• Medical and Population Genetics: Facilitates analysis of population-scale sequencing data for studies in medical genetics and population genomics.
• Personalized Medicine: Accelerates variant discovery workflows that inform personalized medicine initiatives.

Methodology:

Uses Apache Spark to distribute computational tasks and parallelize GATK HaplotypeCaller across cores and nodes; implements adaptive data segmentation to balance workload distribution; and customizes Hadoop-BAM to partition BAM files into overlapped blocks for accurate variant detection.