TSSNote-CyaPromBERT

TSSNote-CyaPromBERT predicts promoter regions in cyanobacterial genomes using BERT-family transformer models adapted to genomic sequences to support analysis of gene regulation.

Key Features:

• Utilization of SOTA NLP Models: Employs XLNET, BERT, and a DNABERT variant (pretrained on the human genome) adapted to treat DNA sequences as a k-mer "language."
• High Predictive Performance: Achieved AUROC 0.97 and F1 0.92 on promoters from Synechococcus elongatus sp. UTEX 2973 and AUROC 0.96 and F1 0.91 in cross-validation with promoters from Synechocystis sp. PCC 6803.
• Custom Pipeline for Model Training: Extracts large datasets from public direct RNA sequencing (dRNA-seq) resources to assemble promoter and non-promoter sequence training sets.
• Transfer Learning Capabilities: Leverages transfer learning of large language models to identify promoter regions in newly isolated cyanobacterial strains with similar lineages.
• Visualization Tools: Provides visualization methods for feature analysis and interpretability of model predictions.

Scientific Applications:

• Promoter annotation: Prediction and annotation of promoter regions in cyanobacterial genomes, including Synechocystis sp. PCC 6803 and Synechococcus elongatus sp. UTEX 2973.
• Gene regulation studies: Analysis of promoter architecture to support investigations of transcriptional regulation in cyanobacteria.
• Synthetic biology and metabolic engineering: Informing design of regulatory elements for phototrophic production of value-added compounds from CO2.
• Cross-strain promoter discovery: Applying transfer learning to annotate promoters in newly isolated or related cyanobacterial strains.

Methodology:

Extracts datasets from public dRNA-seq resources; adapts and fine-tunes XLNET, BERT, and DNABERT (DNABERT pretrained on the human genome) on DNA k-mer tokens; trains models on promoter and non-promoter sequences and evaluates performance via cross-validation using promoters from Synechococcus elongatus sp. UTEX 2973 and Synechocystis sp. PCC 6803.