跳到主要导航 跳到搜索 跳到主要内容

Cross-Modality Image Matching Network With Modality-Invariant Feature Representation for Airborne-Ground Thermal Infrared and Visible Datasets

科研成果: 期刊稿件 › 论文 › 同行评审

65 引用 (Scopus)

摘要

Thermal infrared (TIR) remote-sensing imagery can allow objects to be imaged clearly at night through the long-wave infrared, so that the fusion of thermal infrared and visible (VIS) imagery is a way to improve the remote-sensing interpretation ability. However, due to the large radiation difference between the two kinds of images, it is very difficult to match them. One of the most important issues is the lack of comprehensive consideration of the modality-specific information and modality-shared information, which makes it difficult for the existing methods to obtain a modality-invariant feature representation. In this article, a cross-modality image matching network, which we refer to as CMM-Net, is proposed to realize thermal infrared and visible image matching by learning a modality-invariant feature representation. First, in order to extract the modality-specific features of the imagery, the framework constructs a shallow two-branch network to make full use of the modality-specific information, without sharing parameters. Second, in order to extract the high-level semantic information between the different modalities, modality-shared layers are embedded into the deep layers of the network. In addition, three novel loss functions are designed and combined to learn the modality-invariant feature representation, that is, the discriminative loss of the non-corresponding features in the same modality, the cross-modality loss of the corresponding features between different modalities, and the cross-modality triplet (CMT) loss. The multimodal matching experiments conducted with ground- and airborne-based thermal infrared images and visible images showed that the proposed method outperforms the existing image matching methods by about 2% and 6% for the ground and airborne images, respectively.
源语言英语
页数14
期刊IEEE Transactions on Geoscience and Remote Sensing
卷60
早期在线日期8月 2021
DOI
出版状态已出版 - 1月 2022

基金资助

This work was supported by the National Natural Science Foundation of China under Grant 41771385, Grant 41801267, and Grant 42071350. (Corresponding author: Ailong Ma.)

关键词

  • Image matching
  • Feature extraction
  • Remote sensing
  • Thermal conductivity
  • Task analysis
  • Semantics
  • Training
  • Convolutional neural network (CNN)
  • cross-modality matching
  • deep learning
  • image matching
  • modality-invariant feature representation
  • thermal infrared (TIR) imagery
  • visible (VIS) imagery

收录

  • SCIE
  • EI
  • Scopus

学术指纹

探究 'Cross-Modality Image Matching Network With Modality-Invariant Feature Representation for Airborne-Ground Thermal Infrared and Visible Datasets' 的科研主题。它们共同构成独一无二的学术指纹。

引用此