LCEL

LCEL performs multi-label classification of COVID-19 (LitCovid) literature to extract semantic topics using an ensemble of transformer-based pretrained models.

Key Features:

• Ensemble learning: Integrates multiple transformer-based pretrained models with independent initialization and fine-tuning to improve semantic topic extraction performance.
• Biomedical knowledge integration: Incorporates diverse biomedical knowledge sources to enrich semantic representations of literature inputs.
• Data augmentation: Applies simple data augmentation strategies to increase training sample diversity and enhance model robustness.
• Asymmetric loss function: Uses an asymmetric loss that applies different exponential decay factors to negative versus positive labels to mitigate imbalanced label distributions.
• Ensemble bagging: Consolidates outputs from individual models via an ensemble bagging strategy to produce robust final predictions.
• Seven-model ensemble: Combines seven distinct transformer-based pretrained models to leverage complementary strengths across architectures.

Scientific Applications:

• Automated semantic topic extraction: Enables automatic multi-label classification of COVID-19-related biomedical literature (LitCovid) to support large-scale literature synthesis and reduce manual curation effort.

Methodology:

Integration of seven transformer-based pretrained models with independent initialization and fine-tuning; incorporation of biomedical knowledge sources into model inputs; application of data augmentation; use of an asymmetric loss with different exponential decay factors for label imbalance; and consolidation of model outputs via ensemble bagging.