CCNet

CCNet captures full-image contextual information for visual understanding tasks such as semantic segmentation and object detection.

Key Features:

• Criss-cross attention module: Operates on a per-pixel basis to gather contextual data from all pixels along each pixel's criss-cross path and uses a recurrent operation so each pixel can capture dependencies across the entire image.
• GPU Memory Efficiency: Requires approximately 11 times less GPU memory than the non-local block approach.
• High Computational Efficiency: Employs a recurrent criss-cross attention mechanism that reduces floating-point operations (FLOPs) by about 85% compared to non-local blocks.
• Category consistent loss: Incorporates a category consistent loss to enhance the discriminative power of features produced by the criss-cross attention module.
• State-of-the-art performance: Achieves mean Intersection over Union (mIoU) scores of 81.9% on the Cityscapes test set, 45.76% on the ADE20K validation set, and 55.47% on the LIP validation set.

Scientific Applications:

• Semantic segmentation: Improves contextual modeling for pixel-level labeling, demonstrated by higher mIoU on Cityscapes, ADE20K, and LIP benchmarks.
• Object detection: Captures full-image context to support object-level prediction and localization.

Methodology:

Uses a criss-cross attention module that gathers per-pixel contextual information along criss-cross paths and applies a recurrent operation to propagate dependencies across the image, incorporates a category consistent loss, and is evaluated against non-local blocks reporting ~11× GPU memory reduction and ~85% FLOPs reduction.