Text2Brain

Text2Brain synthesizes 3D brain activation maps from free-form text using a transformer-based neural language model and coordinate-based meta-analysis to enable interpretation of cognitive processes.

Key Features:

• Transformer-Based Neural Network: Uses a transformer-based neural language model to process variable-length text snippets and generate corresponding 3D brain activation maps.
• Coordinate-Based Meta-Analysis Integration: Incorporates coordinate-based meta-analysis of neuroimaging studies drawing from approximately 13,000 published studies.
• Text Encoder and 3D Image Generator: Integrates a transformer-based text encoder with a 3D image generator to translate open-ended textual descriptions into neural activation patterns.
• Training Data Pairing: Trained on diverse text snippets paired with their corresponding brain activation maps sourced from a large corpus of neuroimaging studies.

Scientific Applications:

• Hypothesis Generation: Visualizes potential neural correlates of described cognitive processes to generate new hypotheses.
• Meta-Analytic Augmentation: Enhances meta-analytic capabilities by synthesizing activation maps from textual descriptions beyond keyword-based searches.
• Literature-to-Map Translation: Translates free-form neuroimaging literature text into synthesized brain activation maps for exploratory analysis.

Methodology:

Training of a transformer-based text encoder and 3D image generator on paired text snippets and brain activation maps; incorporation of coordinate-based meta-analysis data from ~13,000 neuroimaging studies.