COMPSs

COMPSs enables execution of parallel computational workflows in Python on distributed infrastructures for scalable scientific computing.

Key Features:

• Sequential Programming Model: Allows writing computational scripts in sequential Python while leveraging Python scientific libraries.
• Annotations/Hints for Parallelism: Functions can be annotated to be executed as asynchronous parallel tasks.
• Automatic Task Dependency Graph Creation: The runtime detects data dependencies and constructs a task dependency graph on-the-fly to manage execution order and concurrency.
• Portability Across HPC and Cloud Environments: Workflows can be executed on high-performance computing and cloud infrastructures without modifying the code.
• Integration with Big Data Storage Architectures: Supports data abstraction as persistent objects accessible from the application layer to interact with large datasets.

Scientific Applications:

• Genomic data analysis: Scales execution of genomic analysis workflows that require distributed computation.
• Protein structure prediction: Executes computationally intensive protein modelling and structure prediction workflows across distributed resources.
• High-throughput bioinformatics workflows: Runs other computationally demanding tasks in bioinformatics and life sciences that benefit from distributed parallelism.

Methodology:

Users write sequential Python scripts and annotate functions as asynchronous tasks; the runtime interprets annotations, automatically detects data dependencies to build a task dependency graph, manages and schedules task execution across distributed resources, and uses persistent objects to abstract data storage.