Foundry

Foundry provides a message-oriented, horizontally scalable Extract-Transform-Load (ETL) system for integrating, transforming, and enhancing biomedical and other scientific datasets to improve data discovery, retrieval, and pooled reanalysis.


Key Features:

  • Message-oriented architecture: Message-oriented architecture for scientific data integration workflows.
  • Horizontal scalability: Distributed, horizontally scalable architecture for handling large volumes of scientific data.
  • Extract-Transform-Load (ETL): An ETL system tailored for scientific data integration and enhancement.
  • Data aggregation and transformation: Aggregates, transforms, and enhances scientific data from diverse sources.
  • Specialized data transformation language: Transforms data using a specialized data transformation language.
  • Unified format loading: Loads transformed data into a unified format to support downstream analyses and discovery.
  • Diverse biomedical dataset support: Accommodates the diverse nature of biomedical datasets for seamless integration.

Scientific Applications:

  • Biomedical research: Enables discovery, retrieval, pooling, and reanalysis of heterogeneous biomedical datasets.
  • Earth sciences: Supports integration, discovery, and pooled analyses of earth sciences datasets.

Methodology:

Foundry employs a distributed ETL process that extracts data from various sources, transforms it using a specialized data transformation language, and loads it into a unified format within a message-oriented, horizontally scalable architecture.

Topics

Details

License:
Other
Maturity:
Mature
Cost:
Free of charge
Tool Type:
command-line tool
Operating Systems:
Linux, Mac
Programming Languages:
Java
Added:
8/11/2019
Last Updated:
6/16/2020

Operations

Publications

Ozyurt IB, Grethe JS. Foundry: a message-oriented, horizontally scalable ETL system for scientific data integration and enhancement. Database. 2018;2018. doi:10.1093/database/bay130. PMID:30576493. PMCID:PMC6301337.

PMID: 30576493
PMCID: PMC6301337
Funding: - National Institute of Health’s Big Data to Knowledge: U24AI117966 - Community Inventory of EarthCube Resources for Geosciences Interoperability: 1343816, 1639764 - National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) Information Network: U24DK097771 - Neuroscience Information Framework via NIH’s National Institute of Drug Abuse: U24DA039832 - Repronim via NIH’s National Institute of Biomedical Imaging and Bioengineering: P41EB019936

Documentation

Links