CheXT

CheXT integrates a radiomics-guided global-local transformer to perform weakly supervised localization and classification of cardiopulmonary pathologies in chest X-rays using radiomic features and image-level disease labels.

Key Features:

• Radiomics-Guided Transformer (RGT) Architecture: A dual-branch architecture that synergizes global image information with local radiomics-guided auxiliary data.
• Image Transformer Branch: Uses self-attention mechanisms to extract image features and propose bounding boxes for pathology localization.
• Radiomics Transformer Branch: Processes computed radiomic features extracted from the proposed regions.
• Fusion Layers: Aggregates information from both image and radiomic branches to integrate diverse data sources.
• Cross-Attention Mechanism: Implements cross-attention layers to enable interaction between learned image and radiomic features using only image-level disease labels.
• End-to-End Feedback Loop: Provides iterative refinement of pathology localization without requiring explicit bounding box annotations.

Scientific Applications:

• Disease Localization: Weakly supervised localization of cardiopulmonary pathologies in chest X-rays.
• Disease Classification: Improved chest X-ray disease classification by integrating domain-specific radiomic features with image features.

Methodology:

CheXT implements a Radiomics-Guided Transformer with Image and Radiomics Transformer branches employing self-attention, cross-attention, fusion layers, and an end-to-end feedback loop, and is trained on the NIH ChestXRay dataset.