omnipy

Omnipy is a Python library for type-driven data wrangling and scalable data flow orchestration. It simplifies creation and deployment of (meta)data transformation processes, emphasizing FAIRification, executable metadata crosswalks and data brokering. Omnipy aids generic data tasks like extraction from multiple sources, (meta)data mapping and systematic data model transformations. Embracing a "parse, don't validate" approach, omnipy uses Python type hints and pydantic for data consistency and adherence to specific models. Highly modular, it allows integration of (meta)data transformation steps dependent on software like pandas and R. The architecture supports pluggable execution engines like Prefect, a highly interoperable industry-developed Open Source data flow orchestration engine. Envisioned as a community effort, omnipy aims to offer various import formats and data models (both hierarchical and tabular representations), as well as ontology mapping and semiautomatic data cleaning.