clustermq

clustermq parallelizes and optimizes the distribution of large numbers of computational tasks on high-performance computing (HPC) clusters to accelerate large-scale bioinformatics analyses such as genomic association studies of drug sensitivity in cancer cell lines.

Key Features:

• Scalability: Manages large numbers of parallelizable tasks without the bottlenecks that limit other job submission packages.
• Performance: Reduces processing overhead and can enable up to three orders of magnitude faster execution compared with previous alternatives when scaling to high task counts.
• Versatility: Applies to a wide range of parallelizable workflows in bioinformatics and is demonstrated for genomic associations of drug sensitivity in cancer cell lines.

Scientific Applications:

• Genomic association studies: Facilitates genomic associations with drug sensitivity in cancer cell lines by enabling large-scale parallel computation.
• High-throughput bioinformatics workflows: Supports large-scale genomic analyses and other parallelizable workflows that require extensive HPC resources.
• Personalized medicine and cancer research: Accelerates analyses relevant to personalized medicine and cancer treatment strategy development.

Methodology:

Optimizes task distribution across HPC clusters to minimize overhead and maximize throughput, using an advanced scheduling mechanism to manage job submissions and executions for improved resource utilization.