DatView

DatView provides visualization and analysis of large tabular datasets to support serial femtosecond crystallography (SFX) data interpretation and processing.

Key Features:

• Tabular dataset visualization: Visualizes large tabular datasets including unmerged SFX indexing results from hundreds of thousands of microcrystal diffraction patterns.
• Parameter plotting: Plots parameters to explore correlations and identify outliers.
• Synchronized selections: Propagates selections across multiple plots to link correlated subsets of data.
• Item viewer: Displays images from the current selection alongside their associated metadata.
• CrystFEL indexamajig integration: Parses indexamajig stream files from CrystFEL and optionally linked HDF5 files to extract parameters and metadata.
• Parameter and metadata tables: Generates comprehensive tables of parameters and metadata derived from stream and HDF5 files.
• Statistics computation: Computes statistics on large datasets to summarize and quantify parameter distributions.
• Comparison mode: Compares indexing and integration parameter sets to assess relative performance.
• Data filtering and export: Supports flexible filtering (plot selections, Boolean filters) and direct export of subsets of CrystFEL stream files for further processing.

Scientific Applications:

• Feedback during data collection: Examines trends over time or across sample positions and other parameters to inform ongoing SFX data collection.
• Optimization of indexing and integration: Enables direct comparison of indexing and integration parameter sets to determine effective parameters.
• Detection of systematic errors: Identifies systematic errors in unmerged SFX datasets that may affect data quality or interpretation.
• Advanced filtering and subset export: Produces filtered subsets of CrystFEL stream files for downstream processing and analysis.

Methodology:

Parses CrystFEL indexamajig stream files and optionally linked HDF5 files to generate parameter and metadata tables, compute dataset statistics, support comparison of indexing/integration parameter sets, and export selected subsets of stream files.