Skip to main navigation Skip to search Skip to main content

Commentary Master: Exploring Fine-grained Video Action Commentary

  • Sifan Zhang
  • , Bingcheng Dong
  • , Yuning Ding
  • , Jinrong Zhang
  • , Qiang Gao
  • , Shenglan Liu*
  • , Tao Sun
  • *Corresponding author for this work
  • Dalian University of Technology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Commentators can provide rich information and analysis to bring good experience to the audience, leading to great application prospects for AI commentators. In this paper, we propose Fine-grained Video Action commentary (FVAC) as a new challenging task that uses long natural language to comment the action in the input video like a real commentator. In addition to describing the action, FVAC focuses on providing detailed evaluation, background knowledge, and technical instructions. Due to the high difficulty to obtain detailed semantic information in the video and the lack of background knowledge, existing researches are based only on action evaluation and description. Fortunately, the great achievements of AI agent in general artificial intelligence give us new solutions. To solve the task, we propose a strong AI agent called the Commentary Master framework to generate fine-grained commentary. In the agent, we propose Discriminator Net (DNet) to extract detailed semantic information and Balance Tree Retrieval-Augmented Generation (BTRAG) to provide rich background knowledge and examples. Additionally, to address the difficulty of evaluating the commentary content with rich styles, we propose the Commentary Quality Level (CQL) metric. Compared to base LVLM models, our experiment shows that the Commentary Master can generate high-quality fine-grained commentary.
Original languageEnglish
Title of host publication8TH INTERNATIONAL ACM WORKSHOP ON MULTIMEDIA CONTENT ANALYSIS IN SPORTS, MMSPORTS 2025
PublisherACM SIGMM
Pages47-64
Number of pages18
ISBN (Electronic)979-8-4007-1835-9
ISBN (Print)9798400711985
DOIs
StatePublished - 27 Oct 2025

Publication series

NameCommentary Master: Exploring Fine-grained Video Action Commentary

Funding

National Natural Science Foundation of China, NNSFC, (62376052); National Natural Science Foundation of China, NNSFC

Indexed by

  • Scopus
  • CPCI-S
  • CPCI-SSH

Fingerprint

Dive into the research topics of 'Commentary Master: Exploring Fine-grained Video Action Commentary'. Together they form a unique fingerprint.

Cite this