Abstract
Image-text multimodal learning in artificial intelli-gence-based fetal cardiac ultrasound imaging analysis is hindered by instance-level alignment. Moreover, instance-level alignment performs poorly in scenarios with scarce annotations and noisy text, which has greatly limited the development of this field. To address this, this paper proposes Prototype-based Category-Level Image-Text Multimodal Learning (PCLIM), which integrates: (1) Prototype-based Labeled Image-Text Contrastive Learning that refines image-text features via Self & Cross-Attention Fusion and optimizes prototype pools through prototype contrastive loss; (2) Dynamic multi-prototype representation using dual-modality prototype pools updated online via EMA and memory buffers to capture evolving intra-class semantics; and (3) Prototype-matched Unlabeled Multimodal Learning that employs an EMA teacher-student framework to match unlabeled samples to category-level pseudo-text anchors with consistency regularization and contrastive learning. Experiments on fetal cardiac ultra-sound datasets EP-CHDT8000 and SYF -CHD30000 demonstrate that PCLIM consistently outperforms state-of-the-art baselines, especially in low-label scenarios. The code is publicly available at https://github.com/SIGMACX/PCLIM. © 2025 IEEE.
| Original language | English |
|---|---|
| Pages | 847-852 |
| DOIs | |
| State | Published - Jan 2025 |
| Event | 2025 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2025 - Wuhan, China, Wuhan, China Duration: 15 Dec 2025 → 18 Dec 2025 |
Conference
| Conference | 2025 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2025 |
|---|---|
| Country/Territory | China |
| City | Wuhan |
| Period | 15/12/25 → 18/12/25 |
Funding
- [2023BAA017]; China Postdoctoral Science Foundation, Number: -, Acronym: -, China Postdoctoral Science Foundation, Number: 2023YFC2705700, Acronym: NKRDPC, National Key Research and Development Program of China, Number: -, Acronym: NKRDPC, National Key Research and Development Program of China, Number: 62306217,62225113, Acronym: NNSFC, National Natural Science Foundation of China, Number: -, Acronym: NNSFC, National Natural Science Foundation of China, Number: GZC20231987, Acronym: -, - [2024M752471,2024T170686]; - [2024AFA017];
Indexed by
- Scopus
- EI
Fingerprint
Dive into the research topics of 'PCLIM: Prototype-Based Category-Level Image-Text Multimodal Learning Method for Fetal Cardiac Ultrasound Imaging Analysis'. Together they form a unique fingerprint.Cite this
- APA
- Author
- BIBTEX
- Harvard
- Standard
- RIS
- Vancouver