Foundry
Foundry provides a message-oriented, horizontally scalable Extract-Transform-Load (ETL) system for integrating, transforming, and enhancing biomedical and other scientific datasets to improve data discovery, retrieval, and pooled reanalysis.
Key Features:
- Message-oriented architecture: Message-oriented architecture for scientific data integration workflows.
- Horizontal scalability: Distributed, horizontally scalable architecture for handling large volumes of scientific data.
- Extract-Transform-Load (ETL): An ETL system tailored for scientific data integration and enhancement.
- Data aggregation and transformation: Aggregates, transforms, and enhances scientific data from diverse sources.
- Specialized data transformation language: Transforms data using a specialized data transformation language.
- Unified format loading: Loads transformed data into a unified format to support downstream analyses and discovery.
- Diverse biomedical dataset support: Accommodates the diverse nature of biomedical datasets for seamless integration.
Scientific Applications:
- Biomedical research: Enables discovery, retrieval, pooling, and reanalysis of heterogeneous biomedical datasets.
- Earth sciences: Supports integration, discovery, and pooled analyses of earth sciences datasets.
Methodology:
Foundry employs a distributed ETL process that extracts data from various sources, transforms it using a specialized data transformation language, and loads it into a unified format within a message-oriented, horizontally scalable architecture.
Topics
Details
- License:
- Other
- Maturity:
- Mature
- Cost:
- Free of charge
- Tool Type:
- command-line tool
- Operating Systems:
- Linux, Mac
- Programming Languages:
- Java
- Added:
- 8/11/2019
- Last Updated:
- 6/16/2020
Operations
Publications
Ozyurt IB, Grethe JS. Foundry: a message-oriented, horizontally scalable ETL system for scientific data integration and enhancement. Database. 2018;2018. doi:10.1093/database/bay130. PMID:30576493. PMCID:PMC6301337.
PMID: 30576493
PMCID: PMC6301337
Funding: - National Institute of Health’s Big Data to Knowledge: U24AI117966
- Community Inventory of EarthCube Resources for Geosciences Interoperability: 1343816, 1639764
- National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) Information Network: U24DK097771
- Neuroscience Information Framework via NIH’s National Institute of Drug Abuse: U24DA039832
- Repronim via NIH’s National Institute of Biomedical Imaging and Bioengineering: P41EB019936
Documentation
Links
Issue tracker
https://github.com/biocaddie/Foundry-ES/issues