Neighbor2Neighbor

Neighbor2Neighbor employs self-supervised neighbor sub-samplers to generate training pairs from noisy images and train denoising networks for image-denoising tasks, including raw Bayer sensor data.

Key Features:

• Self-Supervised Learning: Generates training image pairs solely from noisy inputs using neighbor sub-samplers, enabling network training without clean reference images and achieving performance comparable to supervised methods.
• Theoretical Foundation: Provides theoretical analysis showing that appropriately designed neighbor sub-samplers permit effective training of denoising networks using only noisy images.
• Regularized Loss Function: Incorporates an optimization-based regularizer that minimizes the discrepancy between self-supervised and traditionally supervised denoisers.
• Simple Yet Effective Training Scheme: Trains networks by generating random neighbor sub-sampler image pairs and optimizing with the regularized loss function.
• BayerEnsemble Strategy: Adapts the training strategy for raw image denoising to handle the complexities of Bayer-pattern sensor data.

Scientific Applications:

• Medical Imaging: Enhances clarity of medical scans to support diagnostic tasks.
• Remote Sensing: Improves satellite and aerial imagery for environmental monitoring and analysis.
• Photography and Videography: Refines visual content from consumer electronics such as smartphones and cameras.

Methodology:

Generate training image pairs using random neighbor sub-samplers from noisy images; train denoising networks with a regularized loss that aligns self-supervised and supervised denoisers; and apply the BayerEnsemble adaptation for raw image denoising.