R2IF: Aligning Reasoning with Decisions via Composite Rewards for Interpretable LLM Function Calling
Aijia Cheng, Kailong Wang, Ling Shi, Yongxin Zhao · 2026
Read R2IF: Aligning Reasoning with Decisions via Composite Rewards for Interpretable LLM Function Calling by Aijia Cheng, Kailong Wang, Ling Shi, Yongxin Zhao (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-20316.txt.
Introduction
Cheng and three colleagues present R2IF, a reasoning-aware reinforcement learning frame for clear function calling in language models. Its joint reward blends format and correctness rules with a thought-chain gain reward and a spec-change-value reward, tuned by GRPO. On BFCL and ACEBench it beats rivals by up to 34.62 percent, with a positive mean thought-chain gain score. The work lifts both call hit rate and clarity for reliable tool-augmented use.
Catalog record
| CREATOR | Aijia Cheng, Kailong Wang, Ling Shi, Yongxin Zhao |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 45971 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.20316
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
R2IF: Aligning Reasoning with Decisions via Composite Rewards for Interpretable LLM Function Calling Aijia Cheng, Kailong Wang, Ling Shi, Yongxin Zhao Abstract Function calling empowers large language models (LLMs) to interface with external tools, yet existing RL-based approaches suffer from misalignment between reasoning processes and tool-call decisions. We propose R2IF, a reasoning-aware RL framework for interpretable function calling, adopting a composite reward integrating format/correctness constraints, Chain-of-Thought Effectiveness Reward (CER), and Specification-Modification-Value (SMV) reward, optimized via GRPO. Experiments on BFCL/ACEBench show R2IF outperforms baselines by up to 34.62% (Llama3.2-3B on BFCL) with positive Average CoT Effectiveness (0.05 for Llama3.2-3B), enhancing both function-calling accuracy and interpretability for reliable tool-augmented LLM deployment. 1 Introduction Large language models (LLMs) excel at open-domain text generation but lack reliability in real-time structured interactions (e.g., tool invocation) due to static offline training data Liu et al. 2024b ; Liu et al. 2024a ; Wang et al. 2024 . Function calling addresses this by translating natural language queries into precise executable tool APIs, unlocking up-to-date knowledge and complex task-solving capabilities. Early function calling research relied on supervised fine-tuning (SFT) Hao et al. 2025 with annotated tool-use data, but was bottlenecked by scarce high-quality executable data. As training signals and evaluation protocols matured, reinforcement learning (RL) Qian et al. 2025 ; Zhang et al. 2025 emerged as a powerful alternative, directly optimizing functional correctness and enabling accurate function calls across diverse benchmarks. Yet most RL-based methods remain outcome-driven, rewarding only final correctness via AST evaluation—correct calls Patil et al. 2025 do not equate to sound reasoning, as generated reasoning often serves as post-hoc rationalization rather than guiding tool selection or parameter construction. For example, when queried about San Francisco’s weather (Figure 1 ), the baseline GRPO model’s reasoning only identifies #get_current_weather# but ignores parameter format (City, State) and default values, leading to incomplete tool calls. Given function calling’s structural rigor and sensitivity to fine-grained arguments, accuracy alone fails to capture reasoning quality or system robustness. Figure 1: Our method correctly supplements parameter values (e.g., CA, fahrenheit) by aligning reasoning with tool specifications, while the baseline GRPO model fails to address parameter format requirements. Directly rewarding reasoning without grounding in tool schemas is non-trivial. Unlinked to tool specifications and parameter transformations, process rewards prioritize verbosity over utility. Worse, unconstrained reasoning rewards may erode parameter precision by reinforcing plausible but tool-call decision-misaligned rationales, making "rewarding reasoning" insufficient for reliable tool use. Addressing this gap is pivotal to enhancing tool-augmented LLMs’ interpretability and robustness: ungrounded reasoning complicates error diagnosis, causes failures in unseen parameter formats or queries, and undermines trust and generalization. Effective supervision must distinguish between reasoning that merely accompanies correct actions and reasoning that actively supports parameter specification, necessary modifications, and value instantiation—three pillars of reliable tool use. Our Work. In this work, we propose R2IF, a reasoning-aware reinforcement learning framework for interpretable function calling in large language models. We formalize interpretability as the alignment between reasoning and executable tool-call decisions, grounded in parameter specification, modification, and value instantiation. Building on this, we design a composite reward integrating strict format/correctness constraints, a distribution-based Chain-of-Thought Wei et al. 2023 Effectiveness Reward (CER) quantifying reasoning utility, and a Specification-Modification-Value (SMV) reward enforcing parameter-level reasoning alignment. Optimized via Grouped Proximal Policy Optimization (GRPO) Shao et al. 2024 , R2IF provides trajectory-level supervision to discourage correct-but-ungrounded tool calls and strengthen reasoning-tool call decision links. Results and Findings. Across BFCL and ACEBench benchmarks, R2IF achieves top overall accuracy across Qwen2.5 (1.5B/3B/7B) and Llama3.2-3B backbones, with a +34.62% gain on Llama3.2-3B (BFCL) and +15.4% on Qwen2.5-7B (ACEBench). It maintains strong irrelevant tool-rejection performance ( \geq 96% accuracy) while boosting Atomic and Single-turn task gains, confirming balanced improvement over conservative strategies. R2IF’s reasoning effectiveness (ACE) turns positive for 3B/7B models (e.g., 0.05 for Llama3.2-3B on BFCL), outperforming baselines with negative values. Notably, R2IF scales monotonically with model size and generalizes across model families, remaining competitive under real-user Live scenario distribution shifts. Summary of contributions: Our contributions are summarized as follows: • We propose a hybrid reward design that jointly optimizes reasoning quality and final function-call correctness, addressing the misalignment between reasoning processes and tool-call decisions in RL-based function calling. • We introduce a distribution-based Chain-of-Thought Effectiveness Reward (CER), which enhances tool-call stability without relying on subjective reasoning scoring. • We design a parameter-level Specification-Modification-Value (SMV) as an optimization of outcome-oriented RL methods, explicitly supervising parameter constraints, transformations, and value instantiation to improve interpretability and precision. 2 Background and Related Work Figure 2: Overall pipeline of our reward optimization. We (i) compute a binary reward to enforce strict function-call correctness, (ii) compute a CER reward to measure whether the CoT supports the chosen actions, and (iii) compute an SMV reward to align the CoT with ground-truth parameter specification, modification, and value. Formalization. In the function calling task, our goal is to evaluate the model’s ability to deterministically transform user queries into executable actions. The core formalization of this task is as follows: Given a user query Q and a tool set T=\{t_{1},t_{2},...,t_{n}\} , the model performs a mapping function f , producing an ordered output tuple (R,A)=f(Q,T). Here, R represents the reasoning process, which is a text reasoning sequence; A represents the function call result, which can vary depending on the specific scenario, and it may be an empty action \emptyset , a single action, or a set of actions. The specific scenarios are all presented in Table 1 . Table 1: Comparison of task categories in terms of tool availability and invocation patterns. Task Category Tool Set Size |T| Distinct Tools Used # Tool Calls |A| Simple 1 1 1 Multiple >1 1 1 Parallel 1 1 >1 Parallel Multiple >1 \geq 1 >1 Irrelevance \mathbb{N} 0 0 Function call. Early work on LLM function calling primarily relied on supervised fine-tuning (SFT). To mitigate the lack of high-quality data, several studies proposed scalable synthetic data pipelines. Liu et al. 2024b introduce APIGen, which generates verifiable function-calling data via hierarchical checks, while Liu et al. 2024a propose ToolACE, an agentic synthesis framework that expands API coverage and scenario complexity. For standardized evaluation, Patil et al. 2025 present the Berkeley Function Calling Leaderboard (BFCL), which has become a widely adopted benchmark. More recently, reinforcement learning (RL) has been used to frame function calling as a policy optimization problem, directly optimizing correctness under verifiable rewards. Inspired by R1-style reasoning models ( DeepSeek-AI et al. 2025 ) , RL has been adapted to tool use, with studies exploring reward design for tool selection and argument construction under GRPO ( Qian et al. 2025 ) , as well as rule-based RL with binary correctness and format rewards ( Zhang et al. 2025 ) . RL for LLM Reasoning. Reinforcement learning has emerged as a core post-training paradigm for enhancing LLM reasoning by optimizing generation policies with outcome-based feedback and improving long-horizon credit assignment ( Ouyang et al. 2022 ) . In practice, PPO-style optimizers and their variants (e.g., GRPO) are widely used to stabilize and scale reasoning-oriented training ( Schulman et al. 2017 ; Shao et al. 2024 ; DeepSeek-AI et al. 2025 ) . Beyond answer-level rewards, prior work emphasizes process-aware supervision and step-level evaluation to provide denser signals for multi-step