CodeSteer: Symbolic-Augmented Language Models via Code/Text Guidance
Yongchao Chen, Yilun Hao, Yueying Liu, Yang Zhang, Chuchu Fan · 2025
Read CodeSteer: Symbolic-Augmented Language Models via Code/Text Guidance by Yongchao Chen, Yilun Hao, Yueying Liu, Yang Zhang, Chuchu Fan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-04350.txt.
Introduction
Yongchao Chen, Yilun Hao, Yueying Liu, Yang Zhang, and Chuchu Fan teach language models, in a 2025 preprint, when to reason in words and when to write code. Symbolic tasks often need exact computation, which plain text does badly. Yet models rarely switch to code at the right moments. The authors build a guide model that steers a bigger model between the two modes. They first create a benchmark of thirty-seven symbolic tasks with adjustable difficulty. Then they make training data of guidance exchanges.
The guide, CodeSteerLLM, is a tuned mid-size model with checkers that verify symbols and answers. Paired with a leading commercial model, it lifted the average task score sharply. The pair beat the strongest known reasoning models on the same set. The guide also transferred to other big models, boosting each by a wide margin. The authors argue that steering, not scale alone, unlocks symbolic skill.
Catalog record
| CREATOR | Yongchao Chen, Yilun Hao, Yueying Liu, Yang Zhang, Chuchu Fan |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 66 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-04350
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
CodeSteer: Symbolic-Augmented Language Models via Code/Text Guidance Yongchao Chen, Yilun Hao, Yueying Liu, Yang Zhang, Chuchu Fan Abstract Existing methods fail to effectively steer Large Language Models (LLMs) between textual reasoning and code generation, leaving symbolic computing capabilities underutilized. We introduce CodeSteer, an effective method for guiding LLM code/text generation. We construct a comprehensive benchmark SymBench comprising 37 symbolic tasks with adjustable complexity and also synthesize datasets of 12k multi-turn guidance/generation trajectories and 5.5k guidance comparison pairs. We fine-tune the Llama-3-8B model with a newly designed multi-turn supervised fine-tuning (SFT) and direct preference optimization (DPO). The resulting model, CodeSteerLLM, augmented with the proposed symbolic and self-answer checkers, effectively guides the code/text generation of larger models. Augmenting GPT-4o with CodeSteer raises its average performance score from 53.3 to 86.4, even outperforming the existing best LLM OpenAI o1 (82.7), o1-preview (74.8), and DeepSeek R1 (76.8) across all 37 tasks (28 seen, 9 unseen). Trained for GPT-4o, CodeSteer demonstrates superior generalizability, providing an average 41.8 performance boost on Claude, Mistral, and GPT-3.5. CodeSteer-guided LLMs fully harness symbolic computing to maintain strong performance on highly complex tasks. Models, Datasets, and Codes are available at https://github.com/yongchao98/CodeSteer-v1.0 and https://huggingface.co/yongchao98 . Figure 1 : Examples and performance of CodeSteer on guiding LLM code/text generation to integrate symbolic computing. At each interaction with TaskLLM, it reviews current and previous answers, then provides guidance for the next turn. CodeSteer returns final answers when it deems them ready. With CodeSteer, GPT-4o outperforms OpenAI Code Interpreter, o1, and o1-preview models. 1 Introduction While the reasoning and planning capabilities of LLMs have improved significantly ( Wang et al. 2024 ; Chen et al. 2024c ; Li et al. 2023 ) , they still fail in ostensibly simple tasks ( Zhou et al. 2024a ) . Crucially, many tasks in existing benchmarks—such as Blocksworld ( Valmeekam et al. 2024 ) and Game 24 ( Zhou et al. 2023b ) —can be completely solved with code solutions. Text-based reasoning excels at semantic understanding and commonsense inference but is less suited for exact computation, symbolic manipulation, optimization, and algorithmic processing ( Valmeekam et al. 2022 ) . In contrast, symbolic computing via code generation is adept at handling rigorous operations and can easily leverage specialized tools (e.g., equation solvers). In many tasks, prompting LLMs to generate and execute code outperforms purely textual reasoning ( Madaan et al. 2022 ; Liang et al. 2022 ; Chen et al. 2022 ) . A key challenge is guiding LLMs to decide when to rely on textual reasoning versus programmatic solutions, given that most input questions lack explicit cues about which approach is best. Recent OpenAI GPT models address this by providing a Code Interpreter module, allowing the model to iteratively generate and execute code, then further reason with the output ( Achiam et al. 2023 ) . Multi-agent frameworks like AutoGen ( Wu et al. 2023 ) adopt a specialized system prompt to steer LLM for code generation when needed. However, recently Chen et al. 2025 finds that all these existing methods struggle to effectively steer between textual reasoning and code generation, failing to fully leverage symbolic computing capabilities. Our work tries to bridge this gap by developing an assistant framework (CodeSteer) to guide the code/text generation of the LLM solving the task (TaskLLM). By fine-tuning a small model (Llama-3-8B ( Dubey et al. 2024 ) ) to be the assistant, we enable large models (GPT-4o ( Achiam et al. 2023 ) ) to fully leverage symbolic computing via code generation while preserving other capabilities. Recognizing that iterative “executing and exploring” is the most effective way to solve tasks, we build CodeSteer to generate prompts that guide the TaskLLM through multiple turns of interaction before finalizing answers. To achieve a comprehensive evaluation, we gather and develop a benchmark with 37 symbolic tasks, referred as SymBench. On SymBench, augmenting GPT-4o with CodeSteer greatly improves its average performance score from 53.3 to 86.4, even outperforming the current leading pure-text model, OpenAI o1 (82.7) ( Jaech et al. 2024 ) and DeepSeek R1 (76.8) ( Guo et al. 2025 ) . Although trained for GPT-4o, CodeSteer shows great generalizability, delivering an average 41.8 performance gain on Claude-3-5-Sonnet, Mistral-Large, and GPT-3.5. By fully leveraging symbolic computing, CodeSteer-guided LLMs maintain strong performance on highly complex tasks even when o1 fails in all testing cases. Our key contributions are: 1) Developing and publishing SymBench: Prior works by Chen et al. 2025 and Gui et al. 2024 gathered and developed 14 and 31 tasks, respectively, targeting challenges in computation, symbolic manipulation, logic, optimization, spatial reasoning, and constrained planning. However, neither study published the complete code for question/solution synthesis or the full datasets. From these 45 tasks, we select 37 that remain challenging for GPT-4o and redevelop their generation code to produce samples with adjustable complexity. We refer to this newly published benchmark as SymBench. 2) New methods for dataset construction and model fine-tuning of SFT and DPO: We fine-tune Llama-3-8B with the synthesized datasets of 12k multi-turn guidance/generation trajectories (SFT) and 5.5k guidance comparison pairs (DPO). Unlike standard multi-step settings, in CodeSteer’s multi-turn guidance, the TaskLLM outputs a complete answer each turn rather than only at the end. Consequently, we introduce novel components to both the dataset construction and training processes for SFT and DPO, such as data synthesis of dynamic guidance adaptation, emphasis on the final two turns in SFT, comparison score design, and efficient answer sampling in DPO. These modifications result in better performance. Both the final CodeSteer model and created datasets will be released. 3) Symbolic checker and self-answer checker: Observing that TaskLLM frequently produces text-like code that hardcodes answers, neglecting efficient symbolic computation, we introduce a Symbolic Checker to help CodeSteerLLM evaluate code complexity and efficiency. Since most reasoning and planning tasks can be better verified with coding, we add a Self-answer Checker for better judgment of answer correctness of CodeSteerLLM. These two new checkers have been proven to significantly improve the efficiency of dataset synthesis and CodeSteerLLM fine-tuning. 4) Proposed CodeSteer Outperforms Nine Baselines and o1: CodeSteer’s superior performance highlights the importance of enhancing LLM reasoning and planning with symbolic computing. This also demonstrates the potential for steering large models to generate smarter code and text by leveraging specialized smaller models. 2 Symbolic Tasks and SymBench Challenges in Code/Text Choices For tasks requiring computation, symbolic manipulation, logic, optimization, spatial reasoning, and constrained planning, coding-based symbolic computing is often more effective than text-based approaches. However, Chen et al. 2025 found that steering LLM code/text generation poses significant challenges, even in tasks with apparent symbolic characteristics. The main bottlenecks are: 1) Deciding whether code or text is simpler depends on task type, task complexity, and the LLM’s capabilities, which is hard to judge (see Appendix Sec. A ). 2) LLM-generated code often appears as text-like scripts that merely hard-code answers rather than enabling efficient symbolic computation, echoing the phenomenon described in Yang et al. 2024 (see Appendix Sec. B ). SymBench Chen et al. 2025 and Gui et al. 2024 collected 14 and 31 tasks with symbolic factors from various benchmarks such as Suzgun et al. 2022 ; Chen et al. 2024d ; Yao et al. 2024 ; Cobbe et al. 2021