SCALMU: Synthetically-trained Coupling of Adaptive Learned Multiplicative Updates for Hyperspectral-Multispectral Fusion

Jul 16, 2026

HyperSpectral–MultiSpectral (HSI–MSI) fusion reconstructs a high-resolution hyperspectral image from a low-resolution HSI and a high-resolution MSI. Classical coupled nonnegative matrix factorization (CNMF) offers strong physical interpretability but yields weaker performance than deep learning. SCALMU proposes a blind unrolled neural architecture that couples adaptive learnable matrices with CNMF multiplicative updates, preserving nonnegativity and physical meaning. Synthetic supervision is built using the dead leaves model, enabling end-to-end training despite scarce real pairs; experiments show state-of-the-art gains and release of code.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages15

Reading Time38 min

Introduction

HSI–MSI fusion problem and motivation

Related work: model-based and deep learning methods

Proposed SCALMU approach and contributions

SCALMU: Synthetically-trained Coupling of Adaptive Learned Multiplicative Updates for Hyperspectral-Multispectral Fusion Xinxin Xu, Yann Gousseau, Christophe Kervazo, Sad Ladjal arXiv :2605 .30973v2 [ ee ss .IV] 10 Jul 2026 Abstract—HyperSpectral-MultiSpectral Image (HSI-MSI) fusion aims to recover a high-resolution hyperspectral image from a low-resolution HSI and a high-resolution MSI. Classical methods such as Coupled Nonnegative Matrix Factorization (CNMF) benefit from a strong physical interpretability but suffer from inferior results compared to their deep-learning counterparts. To address this limitation, we propose SCALMU (Syntheticallytrained Coupling of Adaptive Learned Multiplicative Updates), a novel blind unrolled neural network architecture that integrates adaptive learnable matrices within the classical framework of CNMF multiplicative updates, improving its results. Due to its architectural proximity with CNMF, the resulting algorithm preserves physical interpretability and nonnegativity constraints. To overcome the scarcity of supervised training data, we generatea synthetic HSI-MSI dataset using the dead leaves model and train SCALMU end-to-end under synthetic supervision. Experiments on several datasets show that SCALMU outperforms state-of-the-art methods and highlights the potential of blind fusion trained with synthetic data. The code is available at [https://github.com/xinxinxu99/SCALMU.git](https://github.com/xinxinxu99/SCALMU.git) Index Terms—Data fusion; unrolling, hyperspectral image; remote sensing, super-resolution, synthetic training data. I. INTRODUCTION HYperSpectral (HSI) and MultiSpectral (MSI) images are three-dimensional data cubes with two spatial and one spectral dimensions. When both modalities are used to acquire the same scene, the resulting images differ in spectral richness and spatial detail. HSIs contain hundreds of contiguous narrow bands, enabling precise material identification and supporting diverse applications such as source separation [1], target detection [2], vegetation monitoring [3], and land cover classification [4] . However, due to physical constraints, thereis a tradeoff between spatial and spectral resolutions, leading in HSIs to a low spatial resolution, which is further amplified in remote sensing due to large sensor-to-scene distances. In contrast, MSIs capture only a few broad spectral bands but at much higher spatial resolution, offering finer spatial structures and sharper details. Combining these complementary modalities, namely the high spectral precision of HSI and the high spatial details of MSI, defines the HSI–MSI fusion problem, also referred to as hyperspectral image super-resolution. The goal is to reconstruct a high-resolution hyperspectral image that preserves both the spectral fidelity of HSI and the spatial richness of MSI, a capability that has become increasingly important for modern remote sensing and environmental applications [5] . The authors are with LTCI, Tlcom Paris, Institut Polytechnique de Paris, 91120 Palaiseau, France (e-mail: [xinxin.xu@telecom-paris.fr](xinxin.xu@telecom-paris.fr); [yann.gousseau@telecom-paris.fr](yann.gousseau@telecom-paris.fr); [christophe.kervazo@telecom-paris.fr](christophe.kervazo@telecom-paris.fr); [said.ladjal@telecom-paris.fr](said.ladjal@telecom-paris.fr)). Existing HSI–MSI fusion methods can be broadly categorized into model-based and deep learning approaches [6] . Traditional model-based techniques, exemplified by the coupled nonnegative matrix factorization (CNMF) [7], decompose the HSI into high spectral-resolution endmembers and leverage MSI-derived high spatial-resolution abundances through a linear unmixing model. This physically interpretable approach ensures spectral fidelity without requiring ground-truth HRHSI data. However, CNMF requires the prior knowledge of degradation operators and suffers from slow convergence through iterative multiplicative updates. More recently, th

FAQs

What problem does SCALMU address in hyperspectral–multispectral fusion?

SCALMU targets hyperspectral image super-resolution, reconstructing a high-resolution hyperspectral image while preserving both spectral fidelity and spatial detail from complementary HSI and MSI inputs.

How does SCALMU relate to classical CNMF?

SCALMU unrolls CNMF multiplicative updates into a learnable network architecture, integrating adaptive learnable matrices while keeping the algorithm close enough to preserve nonnegativity constraints and physical interpretability.

Why does SCALMU rely on synthetic training data, and how is it generated?

Because supervised real HSI–MSI training pairs are scarce, SCALMU trains end-to-end using a synthetic HSI–MSI dataset generated with the dead leaves model, providing controlled spatial and spectral ground-truth for learning.

Jul 16, 2026

Upload by Olivia Brown