Sherlock

Sherlock enables scalable storage, integration, and distributed query-based analysis of large biological datasets for computational biology.

Key Features:

• Supported data types: Handles structured biological data including interactions, localization, and genomic sequences.
• Columnar storage: Converts disparate datasets into Optimized Row Columnar (ORC) format to improve query performance on large files.
• Distributed queries: Executes distributed analytical queries on large files using PrestoDB.
• Containerized deployment: Leverages Docker for packaging and deployment of query and processing components.
• Loader scripts: Provides loader scripts tailored for genomics, interaction, and expression databases to create and manipulate JavaScript Object Notation (JSON) and ORC file formats.

Scientific Applications:

• Large-scale data analytics: Enables distributed analysis of large genomic and interaction datasets to support computational biology studies.
• Data integration: Facilitates integration of heterogeneous public databases and diverse biological data types into a common storage format for downstream analysis.
• Multi-omic dataset management: Supports storage and querying of genomics, interaction, expression, and localization datasets for large-scale bioinformatics projects.

Methodology:

Uses loader scripts for genomics, interaction, and expression databases to convert datasets (e.g., JSON) into Optimized Row Columnar (ORC) and executes distributed analytical queries via PrestoDB within Docker containers.