Exploring the Vertical-Domain Reasoning Capabilities of Large Language Models
Jie Zhou, Xin Chen, Jie Zhang, Zhe Li · 2025
Read Exploring the Vertical-Domain Reasoning Capabilities of Large Language Models by Jie Zhou, Xin Chen, Jie Zhang, Zhe Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-22443.txt.
Introduction
Jie Zhou, Xin Chen, Jie Zhang, and Zhe Li define and measure accounting reasoning, the domain-specific thinking language models need before enterprises can trust them with accounting work. Models are reshaping research and business practice, yet their fit for professional fields stays unclear. Grounding their criteria in the training data characteristics of the GLM model series, the authors build an evaluation system for accounting-oriented reasoning.
They test GLM-6B, GLM-130B, GLM-4, and GPT-4 across a range of accounting reasoning tasks. Prompt engineering improved every model to varying degrees, and GPT-4 showed the strongest overall accounting reasoning. Still, the results say current models fall short of real-world accounting needs, and enterprise-grade deployment will require much more optimization.
Catalog record
| CREATOR | Jie Zhou, Xin Chen, Jie Zhang, Zhe Li |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 31 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-22443
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Exploring the Vertical-Domain Reasoning Capabilities of Large Language Models Jie Zhou, Xin Chen, Jie Zhang, Zhe Li Abstract Large Language Models (LLMs) are reshaping learning paradigms, cognitive processes, and research methodologies across a wide range of domains. Integrating LLMs with professional fields and redefining the relationship between LLMs and domain-specific applications has become a critical challenge for promoting enterprise digital transformation and broader social development. To effectively integrate LLMs into the accounting domain, it is essential to understand their domain-specific reasoning capabilities. This study introduces the concept of vertical-domain accounting reasoning and establishes evaluation criteria by analyzing the training data characteristics of representative GLM-series models. These criteria provide a foundation for subsequent research on reasoning paradigms and offer benchmarks for improving accounting reasoning performance. Based on this framework, we evaluate several representative models, including GLM-6B, GLM-130B, GLM-4, and OpenAI GPT-4, on a set of accounting reasoning tasks. Experimental results show that different prompt engineering strategies lead to varying degrees of performance improvement across models, with GPT-4 achieving the strongest accounting reasoning capability. However, current LLMs still fall short of real-world application requirements. In particular, further optimization is needed for deployment in enterprise-level accounting scenarios to fully realize the potential value of LLMs in this domain. 1 Introduction With the rapid development of artificial intelligence technologies, fields such as big data, supercomputing, brain-inspired intelligence, and large language models (Large Language Models, LLMs), particularly general-purpose natural language models, have attracted widespread attention from both academia and industry. Artificial intelligence has become a new engine for social intelligence and a core driving force for a new round of technological innovation and industrial transformation. It is also a key driver of national innovation capacity and an important strategic component of national security. In 2017, the State Council issued the New Generation Artificial Intelligence Development Plan, which identified artificial intelligence as a major strategic priority, emphasized the importance of strengthening China’s advantages in the AI field, and promoted interdisciplinary exploratory research. The plan encouraged the integration of artificial intelligence with mathematics, engineering, and other disciplines, strengthened foundational research, and supported the application of AI technologies across various sectors. In July 2023, multiple national ministries jointly issued the Interim Measures for the Administration of Generative Artificial Intelligence Services, encouraging innovation and application of generative artificial intelligence technologies across industries and domains, and promoting the establishment of a standardized, orderly, and innovative governance framework. This policy context has created favorable conditions for the integration of LLMs with accounting practice. As LLMs continue to evolve, they are no longer limited to foundational computing power or model-scale competition, but are increasingly driving the emergence of intelligent industries characterized by deep application integration. With the rapid deployment of general-purpose LLMs in vertical domains, how to effectively integrate LLMs into specific professional workflows and release their domain-specific potential has become a key direction for future research and technological development. Compared with narrowly specialized models, general-purpose LLMs demonstrate strong advantages in accounting-related tasks, including richer cross-domain knowledge, enhanced prompt understanding capabilities, and flexible reasoning and inference processes. These strengths enable LLMs to be applied across a wide range of accounting scenarios, such as intelligent financial analysis, tax compliance, auditing support, and management decision making. However, despite rapid exploration and experimentation, effective domain-specific LLMs for accounting have not yet been systematically established, and the practical effectiveness of LLM-based accounting systems remains limited. In existing applications that integrate LLMs with accounting tasks, prompt engineering is commonly adopted as the primary interaction paradigm. This approach enhances model performance by constructing structured prompts or task-specific templates. However, such methods lack in-depth analysis of the internal reasoning mechanisms of LLMs and are therefore difficult to specialize or optimize systematically. Both approaches rely primarily on the general capabilities of LLMs and do not fundamentally address the need for domain-adaptive reasoning. From the perspectives of mathematical reasoning, logical inference, and symbolic computation, the accounting domain places particularly high demands on the reasoning capabilities of LLMs. Based on insights from natural language generation (NLG) and natural language understanding (NLU), this study argues that the core computational capability underlying LLM performance in accounting tasks is reasoning ability. Accordingly, we propose to systematically analyze and evaluate the accounting reasoning capabilities of LLMs. By combining accounting professional knowledge with LLM reasoning paradigms, we construct an evaluation dataset tailored to accounting expertise and analyze experimental results to identify directions for further optimization. The main contributions of this paper are as follows. First, we extend research on general-purpose LLMs to vertical-domain accounting reasoning. We analyze the relationships between LLM reasoning ability, mathematical reasoning, logical inference, and accounting expertise, providing a theoretical foundation for subsequent studies. Second, we design a comprehensive and diverse benchmark dataset for accounting reasoning evaluation. Unlike existing datasets that focus on single financial subdomains, our dataset covers accounting principles, financial statements, cost accounting, and auditing, enabling a more comprehensive assessment of LLM accounting reasoning performance. Third, we evaluate representative LLMs, including GPT-4 and GLM-series models, on the proposed benchmark and analyze their strengths and limitations, offering valuable insights for accounting-related applications. Finally, we explore the integration of domain knowledge with reasoning optimization strategies to improve LLM performance in accounting tasks, providing guidance for future research and accelerating the deployment of intelligent accounting systems in real-world practice. 2 Related Work 2.1 Development of LLMs in the Accounting Domain The successful application of general-purpose large language models (LLMs) in vertical domains such as medicine and law provides important references for their deployment in the accounting domain. In medical applications, LLMs have been used in clinical decision support, medical image interpretation, and medical record generation ( 22 ) . Due to the high interpretability of medical imaging data, LLMs can effectively identify visual patterns and, when combined with natural language generation, improve the transparency and explainability of AI systems, leading to better clinical outcomes ( 5 ) . In the legal domain, LLMs are often deployed at the application level, as legal environments constitute complex human-designed systems characterized by uncertainty and strict regulatory constraints. AI assistance in this domain must comply with legal norms and ethical requirements, which significantly limits the scope of automated reasoning. The accounting domain shares similarities with law in that both rely on explicit rules and structured standards. However, accounting places greater emphasis on quantitative reasoning, numerical accuracy, and consistency across financial records. Unlike medical image-based tasks, accounting tasks require LLMs to retain accounting standards, reason over causal relationships, and perform multi-step logical and numerical inference. As a result, reasoning capability is particularly critical for accounting applications, especially multi-step computational reasoning. Existing research mainly focuses on the following directions. First, some studies aim to improve specific reasoning abilities of LLMs, such as mathematical or logical reasoning ( 6 ) . However, these methods often struggle to integrate accounting-specific knowledge, as they assume that reasoning capability can be enhanced independently of domain expertise. Consequently, such approaches show limited effectiveness in accounting scenarios ( 4 ) . Moreover, there remains a lack of work explicitly targeting accounting reasoning. Second, some studies attempt to improve domain performance by specializing training data or scaling model parameters. Models such as BloombergGPT ( 15 ) and FinGPT ( 14 ) incorporate large-scale financial corpora or increase model