Learning to Present: Inverse Specification Rewards for Agentic Slide Generation

Karthik Ragunath Ananda Kumar * , Tavus Inc., University of Texas at Dallas and Subrahmanyam Arunachalam * , Texas A&M University · 2026

Read Learning to Present: Inverse Specification Rewards for Agentic Slide Generation by Karthik Ragunath Ananda Kumar * , Tavus Inc., University of Texas at Dallas and Subrahmanyam Arunachalam * , Texas A&M University (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-16839.txt.

Introduction

Ananda Kumar and Arunachalam train an agent to make slide decks with reinforcement learning. Their environment lets a language model research a topic, plan content, and build HTML slides through tools. Rewards mix structure checks, render quality, aesthetic and content scores. One reward is novel: a second model tries to recover the original brief from the slides alone. They fine-tune Qwen2.5-Coder-7B with GRPO on expert demonstrations. On 48 business briefs, the tuned 7B model reached 91.2 percent of Claude Opus 4.6 quality. It also improved 33.1 percent over its base model.

Catalog record

CREATORKarthik Ragunath Ananda Kumar * , Tavus Inc., University of Texas at Dallas and Subrahmanyam Arunachalam * , Texas A&M University
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50387
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.16839

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Learning to Present: Inverse Specification Rewards for Agentic Slide Generation Karthik Ragunath Ananda Kumar * , Tavus Inc., University of Texas at Dallas and Subrahmanyam Arunachalam * , Texas A&M University Abstract Automated presentation generation remains a challenging task requiring coherent content creation, visual design, and audience-aware communication. This work proposes an OpenEnv-compatible reinforcement learning environment where Large Language Model (LLM) agents learn to research topics, plan content, and generate professional HTML slide presentations through tool use. We introduce a multi-component reward system combining structural validation, render quality assessment, LLM-based aesthetic scoring, content quality metrics, and an inverse specification reward that measures how faithfully generated slides convey their intended purpose. The inverse specification reward, an “inverse task” where an LLM attempts to recover the original presentation specification from generated slides, provides a holistic quality signal. Our approach fine-tunes a Qwen2.5-Coder-7B model via GRPO, training only 0.5% of parameters on prompts derived from expert demonstrations collected using Claude Opus 4.6. Experiments on 48 diverse business presentation briefs across six models, including Claude Opus 4.6, Claude Sonnet 4.6, Llama 4 Scout, GPT OSS 120B, and base Qwen 7B, demonstrate that our fine-tuned 7B model achieves 91.2% of Claude Opus 4.6’s quality while improving 33.1% over the untuned base model. The six-model comparison reveals that instruction adherence and tool-use compliance, rather than raw parameter count, determine agentic task performance. The divide-and-conquer reward architecture provides interpretable quality assessment across six dimensions, supporting targeted improvements in agentic presentation generation. We contribute SlideRL , an open-source dataset of 288 multi-turn rollout trajectories across all six evaluated models, publicly available at https://huggingface.co/datasets/KarthikRagunathAnandaKumar/sliderl-multi-turn-rollouts . Code is available at https://github.com/pushing-the-frontier/slide-forge-llm . I Introduction The creation of professional presentations is a ubiquitous task in business, education, and research contexts. Despite advances in generative AI, automated slide generation remains challenging because it requires topic research, content structuring, visual design, and audience-aware communication, all coordinated through a multi-step workflow. Recent work in LLM agents has shown strong results in tool use and multi-step reasoning [ 24 , 17 ] . However, training agents for complex creative tasks like presentation generation poses distinct challenges: (1) the action space is large—the agent must select from 14 tools and specify their parameters, (2) quality assessment requires multiple orthogonal criteria, (3) the task demands both factual accuracy and aesthetic appeal, and (4) slides must follow a coherent narrative arc with logical sequencing and temporal flow across the deck. We address these challenges with a reinforcement learning environment that frames presentation generation as a sequential decision-making problem. The environment exposes 14 tools organized into 5 categories—research ( web_search , fetch_url ), content planning ( create_outline , revise_outline ), design ( generate_slide , edit_slide , set_theme ), deck structure ( get_slide_content , delete_slide , reorder_slides , duplicate_slide , insert_slide ), and meta ( review_deck , finalize )—through which the agent progresses across five phases: research, planning, generation, refinement, and finalization. As illustrated in Fig. 1 , this decomposition divides the complex task into manageable phases while employing a reward architecture that evaluates quality across six dimensions. Fig. 1: Architecture of the proposed system. The LLM agent working in the training loop generates tool calls that are executed in the environment, with multi-component rewards guiding policy optimization. Our key contributions are: 1. OpenEnv [ 10 ] -Compatible RL Environment : A reinforcement learning environment with 14 tools across 5 categories, supporting the full presentation creation workflow from research to finalization. 2. Multi-Component Reward System : A reward architecture combining six quality dimensions with configurable weights, allowing interpretable and targeted quality assessment. 3. Inverse Specification Reward (Novel) : A new “inverse task” reward formulation in which an LLM attempts to reconstruct the original specification from the generated slides alone. To our knowledge, this is the first application of input-reconstruction as a reward signal for evaluating holistic coherence and faithfulness in the context of automated slide and presentation generation. 4. Dense Step Rewards : Quality-delta based step rewards that provide dense training signals rather than sparse episode-end rewards. 5. Multi-Format Output via Tool Use : The fine-tuned model learns to trigger appropriate tool calls that produce presentations in multiple output formats (HTML slide decks and PPTX files), enabling downstream consumption across web rendering and traditional presentation software without format-specific training. 6. Expert Trajectory Generation : A pipeline using Claude Opus 4.6 [ 2 , 3 ] to generate high-quality trajectories for GRPO fine-tuning of smaller models. 7. SlideRL Dataset : We open-source a multi-turn rollout dataset of 288 complete trajectories (48 briefs \times 6 models) with per-turn tool calls, environment observations, step rewards, and quality scores, publicly available at https://huggingface.co/datasets/KarthikRagunathAnandaKumar/sliderl-multi-turn-rollouts . II Related Work II-A LLM Agents and Tool Use Recent work on LLM agents has demonstrated effective tool use across a range of tasks [ 24 , 17 , 26 ] . ReAct [ 26 ] introduced the pattern of interleaving reasoning and acting, while Toolformer [ 17 ] showed that LLMs can learn to use tools through self-supervised learning. Our work extends these approaches to presentation generation, where tool use must be coordinated across research, content creation, and design phases. II-B Reinforcement Learning for LLMs RLHF [ 14 ] established the use of human feedback to align LLMs. Subsequent work has explored alternatives including DPO [ 16 ] , GRPO [ 20 ] , and various reward modeling approaches. Our work employs GRPO for its efficiency in fine-tuning with relative rewards, combined with a multi-component reward architecture tailored to this domain. II-C Automated Presentation Generation Prior work on automated presentation generation has focused on extractive slide generation [ 19 ] , document-to-slide pipelines [ 5 ] , and learning-based content selection [ 7 ] . Recent general-purpose LLM systems [ 12 ] show strong generative capability, but presentation-oriented methods still typically lack the structured reward signals needed for systematic improvement. Our work fills this gap with a multi-component reward architecture. II-D LLM-as-Judge for Quality Assessment Recent work has shown that LLMs can serve as reliable evaluators for generated content [ 27 , 8 ] . Our approach extends this idea through the inverse specification reward, which uses an LLM to assess holistic quality by attempting to recover the original task specification from the generated output. III Environment Design III-A Overview The environment implements the OpenEnv [ 10 ] interface with standard reset() and step() methods. The environment maintains state across an episode, tracking research context, outline structure, generated slides, and workflow phase. Episode Lifecycle: 1. RESEARCH : Agent gathers information via web_search , fetch_url 2. PLAN : Agent structures content via create_outline , revise_outline 3. GENERATE : Agent creates slides via generate_slide 4. REFINE : Agent improves via edit_slide , review_deck 5. DONE : Agent calls finalize to end the episode III-B Action Space The action space consists of 14 tools organized into 5 categories, as shown in Table I . TABLE I: Tool categories and descriptions. Category Tools Description Research web_search , fetch_url Info gathering Content create_outline , revise_outline Planning Design generate_slide , edit_slide , set_theme Slide creation Structure get_slide_content , delete_slide , reorder_slides , duplicate_slide , insert_slide Deck ops Meta review_deck , finalize Quality control Each action is specified as a JSON object with tool name and

Read the complete text