CODECHECK

CODECHECK verifies the executability of computational workflows underlying scientific publications to improve reproducibility and transparency during peer review.

Key Features:

• Independent execution: Performs independent execution of computations underpinning scientific publications during peer review.
• Open infrastructure: Leverages open infrastructure and tools to run and assess computational workflows.
• Integration into review processes: Integrates verification steps into review and publication processes across the dimensions of importance, stakeholder involvement, openness, and timing.
• Structured verification: Provides a structured approach to verifying the executability of computational workflows associated with scientific articles.
• Publisher collaboration: Collaborates with academic publishers and conferences to implement executability checks within publication contexts.
• Empirical demonstration: Demonstrated reproducibility through 25 reproductions of diverse scientific publications.
• Non-textual artifact assessment: Assesses accessibility and executability of non-textual research artifacts to enhance transparency.

Scientific Applications:

• Peer-review verification: Enable independent execution of computations during peer review to verify reported results.
• Publication quality control: Support journals and conferences in incorporating executability checks into publication workflows.
• Reproducibility assessment: Provide evidence for reproducibility standards and track reproducibility across computational research.
• Non-textual artifact evaluation: Evaluate the accessibility and executability of computational workflows associated with manuscripts.

Methodology:

Executes computational workflows using open infrastructure and tools to verify their executability, as demonstrated by 25 reproductions of scientific publications.