摘要
Thermal infrared (TIR) remote-sensing imagery can allow objects to be imaged clearly at night through the long-wave infrared, so that the fusion of thermal infrared and visible (VIS) imagery is a way to improve the remote-sensing interpretation ability. However, due to the large radiation difference between the two kinds of images, it is very difficult to match them. One of the most important issues is the lack of comprehensive consideration of the modality-specific information and modality-shared information, which makes it difficult for the existing methods to obtain a modality-invariant feature representation. In this article, a cross-modality image matching network, which we refer to as CMM-Net, is proposed to realize thermal infrared and visible image matching by learning a modality-invariant feature representation. First, in order to extract the modality-specific features of the imagery, the framework constructs a shallow two-branch network to make full use of the modality-specific information, without sharing parameters. Second, in order to extract the high-level semantic information between the different modalities, modality-shared layers are embedded into the deep layers of the network. In addition, three novel loss functions are designed and combined to learn the modality-invariant feature representation, that is, the discriminative loss of the non-corresponding features in the same modality, the cross-modality loss of the corresponding features between different modalities, and the cross-modality triplet (CMT) loss. The multimodal matching experiments conducted with ground- and airborne-based thermal infrared images and visible images showed that the proposed method outperforms the existing image matching methods by about 2% and 6% for the ground and airborne images, respectively.
| 源语言 | 英语 |
|---|---|
| 页数 | 14 |
| 期刊 | IEEE Transactions on Geoscience and Remote Sensing |
| 卷 | 60 |
| 早期在线日期 | 8月 2021 |
| DOI | |
| 出版状态 | 已出版 - 1月 2022 |
基金资助
This work was supported by the National Natural Science Foundation of China under Grant 41771385, Grant 41801267, and Grant 42071350. (Corresponding author: Ailong Ma.)
关键词
- Image matching
- Feature extraction
- Remote sensing
- Thermal conductivity
- Task analysis
- Semantics
- Training
- Convolutional neural network (CNN)
- cross-modality matching
- deep learning
- image matching
- modality-invariant feature representation
- thermal infrared (TIR) imagery
- visible (VIS) imagery
收录
- SCIE
- EI
- Scopus
学术指纹
探究 'Cross-Modality Image Matching Network With Modality-Invariant Feature Representation for Airborne-Ground Thermal Infrared and Visible Datasets' 的科研主题。它们共同构成独一无二的学术指纹。引用此
- APA
- Author
- BIBTEX
- Harvard
- Standard
- RIS
- Vancouver