Abstract
Commentators can provide rich information and analysis to bring good experience to the audience, leading to great application prospects for AI commentators. In this paper, we propose Fine-grained Video Action commentary (FVAC) as a new challenging task that uses long natural language to comment the action in the input video like a real commentator. In addition to describing the action, FVAC focuses on providing detailed evaluation, background knowledge, and technical instructions. Due to the high difficulty to obtain detailed semantic information in the video and the lack of background knowledge, existing researches are based only on action evaluation and description. Fortunately, the great achievements of AI agent in general artificial intelligence give us new solutions. To solve the task, we propose a strong AI agent called the Commentary Master framework to generate fine-grained commentary. In the agent, we propose Discriminator Net (DNet) to extract detailed semantic information and Balance Tree Retrieval-Augmented Generation (BTRAG) to provide rich background knowledge and examples. Additionally, to address the difficulty of evaluating the commentary content with rich styles, we propose the Commentary Quality Level (CQL) metric. Compared to base LVLM models, our experiment shows that the Commentary Master can generate high-quality fine-grained commentary.
| Original language | English |
|---|---|
| Title of host publication | 8TH INTERNATIONAL ACM WORKSHOP ON MULTIMEDIA CONTENT ANALYSIS IN SPORTS, MMSPORTS 2025 |
| Publisher | ACM SIGMM |
| Pages | 47-64 |
| Number of pages | 18 |
| ISBN (Electronic) | 979-8-4007-1835-9 |
| ISBN (Print) | 9798400711985 |
| DOIs | |
| State | Published - 27 Oct 2025 |
Publication series
| Name | Commentary Master: Exploring Fine-grained Video Action Commentary |
|---|
Funding
National Natural Science Foundation of China, NNSFC, (62376052); National Natural Science Foundation of China, NNSFC
Indexed by
- Scopus
- CPCI-S
- CPCI-SSH
Fingerprint
Dive into the research topics of 'Commentary Master: Exploring Fine-grained Video Action Commentary'. Together they form a unique fingerprint.Cite this
- APA
- Author
- BIBTEX
- Harvard
- Standard
- RIS
- Vancouver