FrMLNet

FrMLNet performs pansharpening by using a framelet-based multilevel convolutional neural network to fuse panchromatic (PAN) and multispectral (MS) images and reconstruct high-resolution multispectral images that preserve spatial and spectral information.

Key Features:

• Framelet-based multilevel CNN: A convolutional neural network built on framelet representations tailored for pansharpening tasks.
• Framelet coefficient prediction: Predicts framelet coefficients of high-resolution MS images from PAN and MS inputs instead of directly inferring pixel-space images.
• Feature Embedding Net: Extracts essential features from PAN and MS images to form the basis for fusion.
• Feature Fusion Net: Integrates spatial information from PAN with spectral information from MS via multilevel feature aggregation.
• Framelet Prediction Net: Predicts the framelet coefficients used to reconstruct high-resolution MS images with preserved detail.
• Hybrid residual connections: Incorporates residual connections across subnetworks to improve gradient flow and learning of complex patterns.
• Validation and performance: Evaluated with quantitative and qualitative experiments at reduced- and full-resolution scales, demonstrating improved spatial and spectral fidelity versus state-of-the-art pansharpening methods.

Scientific Applications:

• Pansharpening of PAN and MS data: Fusion of panchromatic and multispectral satellite imagery to produce high-resolution multispectral outputs.
• Satellite image enhancement: Reconstruction of spatially detailed multispectral images for satellite image analysis.
• Preservation of spatial and spectral fidelity: Producing fused images that retain both spatial detail and spectral characteristics for remote sensing research.

Methodology:

Uses a framelet-based multilevel CNN with three subnetworks (Feature Embedding Net, Feature Fusion Net with multilevel feature aggregation, and Framelet Prediction Net) to predict framelet coefficients from PAN and MS inputs, employs hybrid residual connections, and is evaluated by quantitative and qualitative experiments at reduced- and full-resolution scales.