HookNet

HookNet performs semantic segmentation of histopathology whole-slide images by integrating multi-resolution contextual information with high-resolution local features using a multi-branch convolutional architecture.

Key Features:

• Multi-Resolution Approach: Employs concentric patches at multiple resolutions providing different fields of view to capture both global context and fine details.
• Encoder-Decoder Branches: Processes each resolution with distinct encoder-decoder convolutional neural network branches to extract hierarchical features.
• Hooking Mechanism: Combines intermediate representations from different branches with pixel-wise alignment across feature maps to fuse multi-resolution information.
• Framework Design and Training: Includes design and training constraints to maintain precise alignment during the hooking process for high-resolution semantic segmentation.

Scientific Applications:

• Breast cancer multi-class tissue segmentation: Distinguishes multiple tissue types within breast cancer whole-slide images using combined contextual and local information.
• Lung cancer segmentation of tertiary lymphoid structures and germinal centers: Segments complex structures such as tertiary lymphoid structures and germinal centers in lung cancer histopathology images.

Methodology:

Extraction of concentric multi-resolution patches, processing by distinct encoder-decoder CNN branches, fusion via a hooking mechanism that combines intermediate representations with pixel-wise alignment across feature maps, and training under constraints to preserve alignment.