Skip to main navigation Skip to search Skip to main content

MSSDF: Modality-shared self-supervised distillation for high-resolution multi-modal remote sensing image learning

  • Tong Wang
  • , Guanzhou Chen*
  • , Xiaodong Zhang*
  • , Chenxi Liu
  • , Jiaqi Wang
  • , Xiaoliang Tan
  • , Wenchao Guo
  • , Qingyuan Yang
  • , Kaiqi Zhang
  • *Corresponding author for this work

Research output: Contribution to journal › Article › peer-review

3 Scopus citations

Abstract

High-resolution multi-modal remote sensing (RS) images provide rich complementary information for Earth observation, yet the scarcity of high-quality annotated data remains a major obstacle for effective model training. To address this challenge, we propose a Modality-Shared Self-supervised Distillation Framework (MSSDF) that learns discriminative multi-modal representations with minimal reliance on labeled data. Specifically, MSSDF integrates information-aware and cross-modal masking strategies with multi-objective self-supervised learning, enabling the model to capture modality-shared semantics and compensate for missing or weakly labeled modalities. This design substantially reduces the dependence on large-scale annotations and enhances robustness under limited-label regimes. Extensive experiments on scene classification, semantic segmentation, and change detection tasks demonstrate that MSSDF consistently outperforms state-of-the-art methods, particularly when labeled data are scarce. Specifically, on the Potsdam and Vaihingen semantic segmentation tasks, our method achieved mIoU scores of 78.30% and 76.50%, with only 50% train-set. For the US3D depth estimation task, the RMSE error is reduced to 0.182, and for the binary change detection task in SECOND dataset, our method achieved mIoU scores of 47.51 %, surpassing the second by 3 percentage points. In addition, we construct a high-resolution multi-modal remote sensing image dataset named HR-Pairs, which contains 640,000 DOM (Digital Orthophoto Map)-DSM(Digital Surface Model) pairs with a spatial resolution of 0.05 m, providing a new high-quality dataset for multi-modal remote sensing research. Our pretrain code, checkpoints, and HR-Pairs dataset can be found in https://github.com/CVEO/MSSDF.
Original languageEnglish
Article number104006
Number of pages18
JournalInformation Fusion
Volume129
Early online dateNov 2025
DOIs
StatePublished - May 2026

Funding

<b>Funding</b> This work was supported by the National Natural Science Foundation of China [grant number 42101346] ; the Science and Technology Commission of Shanghai Municipality [grant number 22DZ1100800] ; and the China Postdoctoral Science Foundation [grant number 2020M680109] .

Keywords

  • Multimodal data fusion
  • Remote sensing
  • Self-supervised learning
  • High-resolution images
  • Digital surface model

Indexed by

  • SCIE
  • EI
  • Scopus

Fingerprint

Dive into the research topics of 'MSSDF: Modality-shared self-supervised distillation for high-resolution multi-modal remote sensing image learning'. Together they form a unique fingerprint.

Cite this