U2Fusion

U2Fusion implements an unsupervised end-to-end image fusion network that unifies multi-modal (visible-infrared and medical imaging), multi-exposure, and multi-focus fusion by estimating source-image importance and adaptively preserving information for each fusion task.

Key Features:

• Unsupervised end-to-end network: Trains without ground-truth or specially designed metrics to perform image fusion across tasks.
• Unified multi-task framework: Applies a single model to multi-modal (visible-infrared and medical), multi-exposure, and multi-focus image fusion tasks.
• Source-image importance estimation: Estimates the significance of each source image to guide fusion decisions.
• Adaptive information preservation: Determines task-specific degrees of information preservation for each source image.
• Feature extraction and information measurement: Employs advanced feature extraction and information measurement techniques to assess source contributions.
• Adaptive similarity maintenance: Trains the network to maintain adaptive similarity between the fused result and the source images.
• Sequential-task robustness: Avoids degradation of previously learned capabilities when training a single model on different tasks sequentially.
• Benchmark dataset: Includes the aligned infrared and visible image dataset RoadScene as a benchmark for evaluation.

Scientific Applications:

• Visible–infrared image fusion: Combines visible and infrared images to produce fused imagery that preserves complementary information.
• Medical image fusion: Integrates multiple medical imaging modalities for improved diagnostic representation.
• Multi-exposure fusion: Merges images with different exposures to enhance dynamic range and detail.
• Multi-focus fusion: Fuses images taken at different focus settings to produce all-in-focus results.

Methodology:

Estimate the importance of source images; determine corresponding degrees of information preservation; apply feature extraction and information measurement techniques; train an unsupervised end-to-end network to maintain adaptive similarity between fused results and source images without relying on ground-truth or specially designed metrics.