CAMEO

CAMEO evaluates protein structure prediction methods by performing continuous, automated blind assessments using weekly pre-release Protein Data Bank (PDB) sequences to benchmark modeling accuracy and specific structural features.

Key Features:

• Automated Blind Assessments: Conducts fully automated blind prediction evaluations based on weekly pre-release sequences from upcoming PDB releases.
• Frequent Benchmarking: Assesses approximately 100 targets over a five-week period, typically evaluating ~20 targets per week within a four-day prediction window per target.
• Comprehensive Scoring System: Employs multiple scoring schemes to evaluate binding site accuracy, homo-oligomer interface quality, and local model confidence estimates.
• bestSingleTemplate Reference: Provides the bestSingleTemplate reference based on structure superpositions for objective comparison of 3D modeling accuracy.
• Server Performance Statistics: Publishes weekly submission statistics and performance warnings to monitor server outputs and prediction behavior.
• Integration with Protein Model Portal (PMP): Integrates evaluations with PMP to align continuous assessment of modeling servers with aggregated theoretical models and experimental structures.

Scientific Applications:

• Method benchmarking: Continuous evaluation of protein structure prediction algorithms and servers to measure modeling accuracy over time.
• Binding site and ligand assessment: Quantitative benchmarking of binding site accuracy and ligand-relevant local model confidence estimates.
• Oligomer and quaternary structure evaluation: Assessment of homo-oligomer interfaces and quaternary structure quality.
• Reference-based model comparison: Use of bestSingleTemplate and structure superpositions to identify advances in 3D modeling approaches.
• Server validation and update tracking: Monitoring of modeling server performance via weekly statistics and warnings, and alignment of results with PMP records.

Methodology:

Use of weekly pre-release PDB sequences for fully automated blind prediction evaluations with a four-day prediction window per target; assessment of ~100 targets over five weeks (≈20 targets/week); computation of scores for binding site accuracy, homo-oligomer interface quality, and local model confidence; application of the bestSingleTemplate reference based on structure superpositions; publication of weekly submission statistics and performance warnings; integration of evaluation outputs with the Protein Model Portal (PMP).