Animation Needs Attention: A Holistic Approach to Slides Animation Comprehension with Visual-Language Models
Yifan Jiang, Yibo Xue, Yukun Kang, Pin Zheng, Jian Peng, Feiran Wu, Changliang Xu · 2025
Read Animation Needs Attention: A Holistic Approach to Slides Animation Comprehension with Visual-Language Models by Yifan Jiang, Yibo Xue, Yukun Kang, Pin Zheng, Jian Peng, Feiran Wu, Changliang Xu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-03916.txt.
Introduction
Yifan Jiang and co-authors release the first public data set for understanding slide animations with vision-language models. Their 2025 paper notes that animations engage audiences, yet AI slide tools rarely support them and models reason poorly about motion over time. The data set holds thousands of triplets: a text description, an animation file, and a rendered video, spanning every built-in effect of a popular slide program.
The team fine-tune a mid-size vision-language model with a light adaptation method. The tuned model beat two leading commercial systems on text overlap scores and on their own metric, which grades action coverage, order, and detail. Gains held on a hand-made test set, suggesting real temporal reasoning rather than memorized patterns.
Catalog record
| CREATOR | Yifan Jiang, Yibo Xue, Yukun Kang, Pin Zheng, Jian Peng, Feiran Wu, Changliang Xu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 41 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-03916
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Animation Needs Attention: A Holistic Approach to Slides Animation Comprehension with Visual-Language Models Thanks: Corresponding author: Changliang Xu (Email: [email protected]; ORCID: https://orcid.org/0009-0000-2073-3429). Yifan Jiang, Yibo Xue, Yukun Kang, Pin Zheng, Jian Peng, Feiran Wu, Changliang Xu Abstract Slide animations—such as fade-in, fly-in, and wipe—are essential for audience engagement, efficient information delivery, and vivid visual expression. Yet most AI-driven slide-generation tools still lack native animation support, and existing vision–language models (VLMs) underperform on animation tasks because no public datasets exist, and their temporal-reasoning ability remains limited. To close this gap, we release the first public dataset for slide-animation modelling: 12,000 triplets of natural-language descriptions, animation JSON files, and rendered videos that collectively cover every built-in PowerPoint effect. Using this resource, we fine-tune Qwen-2.5-VL-7B with Low-Rank Adaptation (LoRA) and achieve consistent gains over GPT-4.1 and Gemini-2.5-Pro on BLEU-4, ROUGE-L, SPICE, and our Coverage–Order–Detail Assessment (CODA) metric, which jointly gauges action coverage, temporal order, and detail fidelity. On a manually created slides test set, the LoRA model lifts BLEU-4 by ≈ \approx 60 % and ROUGE-L by ≈ \approx 30 %, while posting double-digit improvements in CODA-detail—evidence that low-rank adaptation confers reliable temporal reasoning and generalization beyond synthetic data. Overall, our dataset, LoRA-enhanced model, and CODA metric provide a rigorous benchmark and a preparation for future research on VLM-based dynamic slide generation. Keywords Slide \cdot Animation \cdot Multimodal Data Synthesis \cdot Vision-Language Model \cdot LoRA (Low-Rank Adaptation) 1 Introduction Slides are ubiquitous in education, business, and scientific communication, prized for their clarity and flexibility. Animation effects—such as fade-in, fly-in, and wipe—add temporal dynamics to otherwise static content by precisely controlling the timing, direction, and style of each transition. These effects sharpen information delivery, capture audience attention, and enrich visual expression. Yet, despite the rise of AI-driven agent tools, most slide-generation systems still omit animation, limiting their utility for dynamic presentations. This study tackles that gap by introducing methods for the automatic understanding and generation of slide animations. Animated slides differ fundamentally from plain text or static images: they comprise multi-frame sequences with strict temporal dependencies, demanding sophisticated multimodal reasoning and sequence modeling. Current vision–language models (VLMs) struggle for two main reasons: (1) no public datasets exist that target slide animations, hindering systematic training and evaluation; and (2) existing models have only limited ability to capture fine-grained motion cues and uphold temporal order. Multimodal understanding has advanced rapidly in recent years. Datasets for document visual question answering—such as DocVQA [ 1 ] , InfographicVQA [ 2 ] , and SlideVQA [ 3 ] —have spurred work on static documents and slides, whereas large-scale video-caption corpora (e.g., MSR-VTT [ 4 ] and ActivityNet Captions [ 5 ] ) have accelerated research on dynamic scene description. Yet methods aimed at text-heavy images (e.g., TextVQA [ 6 ] ) still fail to model temporal dynamics, and open-domain video captioners seldom achieve the precision required for slide’s structured layouts and parameterized animations. To close this gap, we release a synthetic dataset of 12,000 triplets—natural-language animation descriptions, animation JSON files, and rendered videos—explicitly created for training and evaluating VLMs on slide-animation understanding. On this resource we fine-tune Qwen 2.5-VL-7B [ 7 ] with Low-Rank Adaptation (LoRA), which adds only a small set of trainable weights yet markedly improves the model’s ability to capture fine-grained motion cues and maintain temporal order—particularly valuable in low-resource settings [ 8 ] . Finally, we propose Coverage–Order–Detail Assessment (CODA) , an LLM-based metric that scores action coverage, temporal order, and detail fidelity, offering a comprehensive evaluation of VLM performance on slide-animation recognition. PowerPoint serves as a practical and widely adopted platform for modeling slide animations due to its rich built-in effects, structured layout system, and native support for timed transitions—all essential for capturing the temporal and visual characteristics of real-world presentations. We develop an end-to-end synthesis pipeline that combines large language models, the python-pptx library [ 9 ] , and Visual Basic for Applications (VBA) to produce the first public dataset of 12,000 text–JSON–video triplets spanning a wide range of PowerPoint animation effects and layouts. Building on this resource, we fine-tune Qwen-2.5-VL-7B with a LoRA-based strategy that integrates multimodal inputs, markedly strengthening the model’s slide-animation understanding. We also introduce Coverage–Order–Detail Assessment (CODA) and show that the LoRA model consistently outperforms baselines on BLEU-4 [ 10 ] , ROUGE [ 11 ] , SPICE [ 12 ] , and all CODA sub-scores, while generalizing well to manually created slides from diverse domains. 1 1 1 Dataset and code will be made available at https://github.com/pampas-lab upon acceptance of the manuscript. The remainder of the paper is structured as follows. Section 2 surveys related work and its limitations for slide-animation recognition. Section 3 describes the dataset construction pipeline and accompanying statistics. Section 4 details our LoRA fine-tuning method. Section 5 presents the experimental setup, CODA metric design, ablation studies, and comparative results. Section 6 concludes the paper and discusses future directions. 2 Related Work 2.1 Visual Language Models and Document Understanding The rapid advancement of Large Language Models (LLMs) has catalyzed significant progress in Visual Language Models (VLMs), enabling sophisticated cross-modal understanding capabilities. CLIP [ 13 ] established foundational work by learning joint representations of images and text through contrastive learning, while BLIP-2 [ 14 ] introduced the Querying Transformer architecture to efficiently bridge visual encoders with language models. Instruction-following VLMs such as LLaVA [ 15 ] and InstructBLIP [ 16 ] have demonstrated enhanced capabilities in following complex visual instructions, while Flamingo [ 17 ] has shown remarkable few-shot learning abilities. The Qwen 2.5-VL-7B model [ 7 ] adopted in this paper is a representative open-source model stemming from this technological trend. It possesses outstanding image-text understanding and conversational abilities, providing a solid foundation for our targeted fine-tuning. In the domain of document understanding, Visual Question Answering (VQA) has emerged as a mature research area with specialized datasets including DocVQA [ 1 ] for scanned documents, InfographicVQA [ 2 ] for complex infographics, and TextVQA [ 6 ] for dense text understanding. The LayoutLM series [ 18 , 19 , 20 , 21 ] has significantly advanced structured document understanding by comprehending spatial relationships and textual content simultaneously, while UDOP [ 22 ] has further unified document understanding through integrated vision, text, and layout processing. Most relevant to our work is SlideVQA [ 3 ] , which targets static PowerPoint slides, requiring understanding of layout, content, and logical relationships within individual slides. However, existing approaches are fundamentally limited to static documents, focusing primarily on spatial layout understanding and OCR capabilities. They lack the ability to process temporal information inherent in PowerPoint animations, including element entrance sequences, motion trajectories, and transition effects—a critical gap our work addresses. 2.2 Video Understanding and Temporal Modeling PowerPoint animations can be conceptualized as specialized short videos, making video captioning research highly relevant. Large-scale datasets such as MSR-VTT [ 4 ] and ActivityNet Captions [ 5 ] have driven progress in dynamic scene understanding, while complex datasets like YouCookII [ 23 ] have advanced step-by-step temporal decomposition in procedural videos. In recent years, video-language models have demonstrated impressive capabilities: Video-ChatGPT [ 24 ] and VideoLLaMA [ 25 ] have shown strong performance in video understanding and dialogue, while X-CLIP [ 26 ]