PDBUSQLExtractor

PDBUSQLExtractor extracts structural data from Protein Data Bank (PDB) files within Azure Data Lake to enable large-scale geometric calculations and interaction analyses of proteins, nucleic acids, and their complexes.

Key Features:

• Dedicated Data Extractors: Custom data extractors parse PDB files and prepare structural information for downstream computation.
• Cloud-Based Scalability: Operates within Azure Data Lake to scale storage and compute for large macromolecular datasets.
• Data Compression: Compresses PDB files to reduce storage footprint while maintaining processing efficiency.
• Parallel Processing: Distributes calculations and data extractions across multiple nodes to accelerate analyses.
• Sequential File Storage: Stores macromolecular data in large sequential files to optimize read/write performance for massive datasets.
• Declarative U-SQL Scripting: Supports declarative U-SQL scripts executed in Azure Data Lake Analytics for defining and running analytical workflows.

Scientific Applications:

• Structural Feature Calculation: Compute geometric properties and high-resolution structural features from PDB data for proteins, nucleic acids, and complexes.
• Interaction Studies: Analyze protein–nucleic acid and protein–protein interactions based on parsed structural data.
• Large-Scale Data Analysis: Support genome-wide and repository-scale studies by processing extensive PDB datasets within a scalable cloud environment.

Methodology:

Parsing PDB files with custom extractors, compressing files, storing data in large sequential files, parallelizing computations across multiple nodes, and executing declarative U-SQL scripts in Azure Data Lake Analytics.