Abstract
High-resolution multi-modal remote sensing (RS) images provide rich complementary information for Earth observation, yet the scarcity of high-quality annotated data remains a major obstacle for effective model training. To address this challenge, we propose a Modality-Shared Self-supervised Distillation Framework (MSSDF) that learns discriminative multi-modal representations with minimal reliance on labeled data. Specifically, MSSDF integrates information-aware and cross-modal masking strategies with multi-objective self-supervised learning, enabling the model to capture modality-shared semantics and compensate for missing or weakly labeled modalities. This design substantially reduces the dependence on large-scale annotations and enhances robustness under limited-label regimes. Extensive experiments on scene classification, semantic segmentation, and change detection tasks demonstrate that MSSDF consistently outperforms state-of-the-art methods, particularly when labeled data are scarce. Specifically, on the Potsdam and Vaihingen semantic segmentation tasks, our method achieved mIoU scores of 78.30% and 76.50%, with only 50% train-set. For the US3D depth estimation task, the RMSE error is reduced to 0.182, and for the binary change detection task in SECOND dataset, our method achieved mIoU scores of 47.51 %, surpassing the second by 3 percentage points. In addition, we construct a high-resolution multi-modal remote sensing image dataset named HR-Pairs, which contains 640,000 DOM (Digital Orthophoto Map)-DSM(Digital Surface Model) pairs with a spatial resolution of 0.05 m, providing a new high-quality dataset for multi-modal remote sensing research. Our pretrain code, checkpoints, and HR-Pairs dataset can be found in https://github.com/CVEO/MSSDF.
| Original language | English |
|---|---|
| Article number | 104006 |
| Number of pages | 18 |
| Journal | Information Fusion |
| Volume | 129 |
| Early online date | Nov 2025 |
| DOIs | |
| State | Published - May 2026 |
Funding
<b>Funding</b> This work was supported by the National Natural Science Foundation of China [grant number 42101346] ; the Science and Technology Commission of Shanghai Municipality [grant number 22DZ1100800] ; and the China Postdoctoral Science Foundation [grant number 2020M680109] .
Keywords
- Multimodal data fusion
- Remote sensing
- Self-supervised learning
- High-resolution images
- Digital surface model
Indexed by
- SCIE
- EI
- Scopus
Fingerprint
Dive into the research topics of 'MSSDF: Modality-shared self-supervised distillation for high-resolution multi-modal remote sensing image learning'. Together they form a unique fingerprint.Cite this
- APA
- Author
- BIBTEX
- Harvard
- Standard
- RIS
- Vancouver