Skip to main navigation Skip to search Skip to main content

PCLIM: Prototype-Based Category-Level Image-Text Multimodal Learning Method for Fetal Cardiac Ultrasound Imaging Analysis

  • Xi Chen
  • , Lyuyang Tong*
  • , Bo Du*
  • *Corresponding author for this work

Research output: Contribution to conference › Conference Paper › peer-review

Abstract

Image-text multimodal learning in artificial intelli-gence-based fetal cardiac ultrasound imaging analysis is hindered by instance-level alignment. Moreover, instance-level alignment performs poorly in scenarios with scarce annotations and noisy text, which has greatly limited the development of this field. To address this, this paper proposes Prototype-based Category-Level Image-Text Multimodal Learning (PCLIM), which integrates: (1) Prototype-based Labeled Image-Text Contrastive Learning that refines image-text features via Self & Cross-Attention Fusion and optimizes prototype pools through prototype contrastive loss; (2) Dynamic multi-prototype representation using dual-modality prototype pools updated online via EMA and memory buffers to capture evolving intra-class semantics; and (3) Prototype-matched Unlabeled Multimodal Learning that employs an EMA teacher-student framework to match unlabeled samples to category-level pseudo-text anchors with consistency regularization and contrastive learning. Experiments on fetal cardiac ultra-sound datasets EP-CHDT8000 and SYF -CHD30000 demonstrate that PCLIM consistently outperforms state-of-the-art baselines, especially in low-label scenarios. The code is publicly available at https://github.com/SIGMACX/PCLIM. © 2025 IEEE.
Original languageEnglish
Pages847-852
DOIs
StatePublished - Jan 2025
Event2025 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2025 - Wuhan, China, Wuhan, China
Duration: 15 Dec 2025 → 18 Dec 2025

Conference

Conference2025 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2025
Country/TerritoryChina
CityWuhan
Period15/12/25 → 18/12/25

Funding

- [2023BAA017]; China Postdoctoral Science Foundation, Number: -, Acronym: -, China Postdoctoral Science Foundation, Number: 2023YFC2705700, Acronym: NKRDPC, National Key Research and Development Program of China, Number: -, Acronym: NKRDPC, National Key Research and Development Program of China, Number: 62306217,62225113, Acronym: NNSFC, National Natural Science Foundation of China, Number: -, Acronym: NNSFC, National Natural Science Foundation of China, Number: GZC20231987, Acronym: -, - [2024M752471,2024T170686]; - [2024AFA017];

Indexed by

  • Scopus
  • EI

Fingerprint

Dive into the research topics of 'PCLIM: Prototype-Based Category-Level Image-Text Multimodal Learning Method for Fetal Cardiac Ultrasound Imaging Analysis'. Together they form a unique fingerprint.

Cite this