BERT-GT

BERT-GT extracts cross-sentence n-ary relations from biomedical text by integrating Bidirectional Encoder Representations from Transformers (BERT) with a Graph Transformer and a neighbor-attention mechanism to improve relation-extraction accuracy.

Key Features:

• Integration of BERT and Graph Transformer: Combines BERT representations with a Graph Transformer / graph neural network architecture to model relationships among multiple entities across sentences.
• Neighbor-Attention Mechanism: Implements neighbor-attention that restricts attention calculations to immediate neighboring tokens to reduce noise and better handle long texts.
• Cross-Sentence N-ary Relation Extraction: Specifically handles n-ary relations that span multiple sentences and abstract-level contexts.
• Targeted Biological Entities: Extracts relations involving genes, diseases, chemicals, and mutations.
• Benchmark Performance: Demonstrates improvements of 5.44% in accuracy and 3.89% in F1-measure on n-ary and chemical-protein relation datasets compared to prior methods.

Scientific Applications:

• Biomedical text mining: Extraction of complex interrelations among biological entities from biomedical literature.
• Genomics, proteomics, and pharmacology research: Support for studies requiring automated extraction of multi-entity relations across sentences in large text corpora.

Methodology:

Combines BERT with a Graph Transformer / graph neural network and applies a neighbor-attention mechanism that limits attention to immediate neighboring tokens for cross-sentence n-ary relation extraction.