Hespi

Hespi extracts authoritative biodiversity data from herbarium specimen sheets using advanced computer vision techniques.

Key Features:

• Object Detection: Hespi integrates two object detection models, one for detecting components of the sheet and another for fields on the primary specimen label.
• Label Classification: Hespi classifies labels as printed, typed, handwritten, or mixed.
• Text Extraction: Hespi uses optical character recognition and handwritten text recognition for text extraction.
• Text Correction: Extracted text is corrected against authoritative taxon databases.
• Text Refinement: Hespi refines extracted text using a multimodal large language model.
• Modular Design: Hespi's modular design allows users to train and integrate custom models.

Scientific Applications:

• Biodiversity Data Extraction: Hespi enables efficient extraction of specimen-associated biodiversity data from herbarium specimen images for biological, environmental, and conservation sciences.
• Cross-Institutional Herbarium Digitization: Hespi accurately detects and extracts text from specimen sheets across international herbaria.

Methodology:

Hespi integrates two object detection models for sheet component and label field detection, classifies labels by type, applies optical character recognition and handwritten text recognition, corrects text against authoritative taxon databases, and refines results using a multimodal large language model.