Acidbio

Acidbio evaluates interoperability of bioinformatics software for genomics file formats by testing input validation and handling of BED variants to improve correctness and reproducibility of genomic analyses.

Key Features:

• Automated testing framework: Evaluates behavior of bioinformatics software packages processing genomics file formats and focuses on edge cases to test input and output handling.
• Manual test design: Uses manually crafted tests to target specific format variants and known edge cases.
• Fuzzing approach: Applies automated fuzzing with random or semi-random data to uncover errors not detected by manual tests.
• Performance analysis: Assessed input validation of 80 Bioconda packages that parse the Browser Extensible Data (BED) format, revealing that only a small fraction met acceptable correctness.
• Root cause categorization: Structures and categorizes causes of software failures to identify recurring implementation issues.
• Badge system: Produces badges indicating which BED variants a software package supports and its performance metrics on the test suite.

Scientific Applications:

• Reliability assessment of parsing and input validation: Quantifies how well tools handle diverse and malformed genomics file inputs, including BED variants.
• Error detection and prevention in genomic analyses: Identifies software behaviors that could lead to undetected errors in downstream analyses.
• Guiding tool development and reproducibility: Provides categorized failure modes and performance metrics to guide developers in improving interoperability and reproducibility of genomic workflows.

Methodology:

Combines manually crafted test cases and automated fuzzing to exercise software on genomics file formats and performs structured analysis to categorize errors and assess input validation performance.