shournal

shournal records shell commands and associated file metadata to document computational workflows and improve reproducibility of data analyses.

Key Features:

• Automatic Command Logging: Automatically logs every shell command executed, including working directories and command text.
• File Metadata Recording: Records metadata for files read or written during shell operations, including file paths and checksums.
• Script and Configuration Archiving: Archives scripts and configuration files that are not under version control to preserve workflow components.
• Low Overhead: Performs logging with low runtime and storage overhead to minimize impact on system performance.

Scientific Applications:

• Reproducibility of Shell-Based Analyses: Provides an automated record of shell activities to enable reproduction of computational analyses performed in Linux environments.
• Data Provenance and Audit Trails: Enables tracking of file operations and integrity verification for data provenance and auditing of computational workflows.

Methodology:

Integrates with the Linux shell to capture every executed command and associated file operations in real time, recording working directories, file paths, file metadata and checksums; archives observed scripts and configuration files; and exports observed shell-command series to Snakemake rules via the external shournal-to-snakemake utility.