Abstract
Establishing correspondences between 2D images and 3D models is essential for precise 3D modeling and accurate positioning. However, widely adopted techniques for aligning 2D images with 3D features heavily depend on dense 3D reconstructions, which not only incur significant computational demands but also tend to exhibit reduced accuracy in texture-poor environments. In this study, we propose a novel method that combines local feature description and detection to enable direct and automatic alignment of 2D images with 3D models. Our approach utilizes a twin convolutional network architecture to process images and 3D data, generating respective feature maps. To address the non-uniform distribution of pixel and spatial point densities, we introduce an ultrawide perception mechanism to expand the receptive field of image convolution kernels. Next, we apply a nonlocal maximum suppression criterion to concurrently evaluate the salience of pixels and 3D points. Additionally, we design an adaptive weight optimization loss function that dynamically guides learning objectives toward sample similarity. We rigorously validate our approach on multiple datasets, and our findings demonstrate successful co-extraction of cross-modal feature points. Through comprehensive 2D-3D feature matching experiments, we benchmark our method against several state-of-the-art techniques from recent literature. The results show that our method outperforms nearly all evaluated metrics, underscoring its effectiveness.
| Original language | English |
|---|---|
| Pages (from-to) | 155-169 |
| Number of pages | 15 |
| Journal | ISPRS Journal of Photogrammetry and Remote Sensing |
| Volume | 229 |
| Early online date | Aug 2025 |
| DOIs | |
| State | Published - Nov 2025 |
Funding
This work was jointly supported by the Young Scientists Fund of the National Natural Science Foundation of China (42301520) , the Early Career Scheme of Hong Kong Research Grants Council (25206524) , the Science and Technology Research Project of Chongqing Education Commission (KJQN202200744) , the Open Research Fund of Shenzhen Key Laboratory of Spatial Smart Sensing and Services (GEMLab-2024001) .
Keywords
- 2D-3D Local Feature Matching
- Cross-Modal Feature Description and Detection
- Twin Convolutional Networks
- Non-Local Maximum Suppression
Indexed by
- SCIE
- EI
- Scopus
Fingerprint
Dive into the research topics of 'Cross-modal 2D-3D feature matching: simultaneous local feature description and detection across images and point clouds'. Together they form a unique fingerprint.Cite this
- APA
- Author
- BIBTEX
- Harvard
- Standard
- RIS
- Vancouver