IndexNet

IndexNet implements index-guided upsampling to improve spatial detail recovery in convolutional neural networks for dense prediction tasks.

Key Features:

• Index function unification: Unifies existing upsampling operators under an index function that guides reconstruction during upsampling.
• Learn-to-index: Adaptively learns indices from the feature map as a function of the data without requiring additional supervision.
• Index Network module: Provides a learnable module that dynamically generates indices conditioned on the feature map.
• Guided downsampling and upsampling: Uses the learned indices to guide both downsampling and upsampling (index-guided unpooling) operations.
• Multiple IndexNet families: Investigates five distinct families of IndexNet architectures for different indexing strategies.
• Boundary-detail recovery: Demonstrates index-guided unpooling outperforms traditional methods such as bilinear interpolation in recovering boundary details.
• Plug-in integration: Functions as a plug-in module for convolutional networks with coupled downsampling and upsampling stages.

Scientific Applications:

• Image Matting: Validated for deep image matting where index-guided upsampling improves boundary recovery.
• Image Denoising: Applied to image denoising as a dense prediction task and validated in experiments.
• Semantic Segmentation: Applied to semantic segmentation for dense prediction and validated in experiments.
• Monocular Depth Estimation: Applied to monocular depth estimation for dense prediction and validated in experiments.

Methodology:

IndexNet unifies upsampling operators via an index function and implements a learn-to-index approach where an Index Network dynamically generates indices conditioned on the feature map; these indices guide downsampling and index-guided unpooling during upsampling, with five IndexNet families evaluated on synthetic data and validated across four dense prediction tasks.