SparkEC

SparkEC accelerates alignment-based DNA error correction for Next Generation Sequencing (NGS) reads by implementing CloudEC algorithms on the Apache Spark distributed framework to provide scalable preprocessing of genomic data.

Key Features:

• Alignment-based error correction: Implements the alignment-based DNA error-correction algorithms established by CloudEC.
• Apache Spark parallel processing: Executes corrections using Apache Spark's distributed computing across multiple cluster nodes.
• Memory-efficient data structures: Employs memory-efficient data structures to reduce resource consumption and increase processing speed.
• No input preprocessing: Omits preprocessing of input reads prior to error correction.
• Scalability: Processing throughput increases proportionally with the number of nodes in a computing cluster.
• Reported performance: Demonstrates average and maximum speedups of 4.9× and 11.9×, respectively, compared to CloudEC across representative datasets.

Scientific Applications:

• Preprocessing of NGS reads: Corrects DNA reads from Next Generation Sequencing platforms to improve data quality for downstream analyses.
• Large-scale genomic datasets: Enables high-throughput error correction of large NGS datasets on distributed computing clusters.

Methodology:

Implements CloudEC's alignment-based error-correction algorithms on Apache Spark using distributed parallel processing and memory-efficient data structures while omitting input preprocessing.