Assessing LLM Reasoning Steps via Principal Knowledge Grounding
Hyeon Hwang Yewon Cho Chanwoong Yoon Yein Park Minju Song, Kyungjae Lee Gangwoo Kim Jaewoo Kang, Korea University University of Seoul AWS AI Labs AIGEN Sciences{hyeon-hwang, yewon330, kangj}@korea.ac.kr · 2025
Read Assessing LLM Reasoning Steps via Principal Knowledge Grounding by Hyeon Hwang Yewon Cho Chanwoong Yoon Yein Park Minju Song, Kyungjae Lee Gangwoo Kim Jaewoo Kang, Korea University University of Seoul AWS AI Labs AIGEN Sciences{hyeon-hwang, yewon330, kangj}@korea.ac.kr (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-00879.txt.
Introduction
Hyeon Hwang and colleagues build an evaluation suite for checking whether a language model's step by step reasoning is truly grounded in knowledge. Models now reason in chains, but a fluent chain can still rest on missing or misused facts.
The suite has three parts: a large collection of atomic knowledge items that reasoning depends on, metrics that score how well a model recalls and applies that knowledge mid chain. A small evaluator model trained to compute those scores cheaply and reliably. The suite proves good at spotting missing or misapplied knowledge, exposing basic reasoning flaws. The authors also show the metrics can feed preference optimization, turning the evaluation into a training signal.
Catalog record
| CREATOR | Hyeon Hwang Yewon Cho Chanwoong Yoon Yein Park Minju Song, Kyungjae Lee Gangwoo Kim Jaewoo Kang, Korea University University of Seoul AWS AI Labs AIGEN Sciences{hyeon-hwang, yewon330, kangj}@korea.ac.kr |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 63 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-00879
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Assessing LLM Reasoning Steps via Principal Knowledge Grounding Hyeon Hwang Yewon Cho Chanwoong Yoon Yein Park Minju Song, Kyungjae Lee Gangwoo Kim Jaewoo Kang, Korea University University of Seoul AWS AI Labs AIGEN Sciences{hyeon-hwang, yewon330, kangj}@korea.ac.kr Abstract Step-by-step reasoning has become a standard approach for large language models (LLMs) to tackle complex tasks. While this paradigm has proven effective, it raises a fundamental question: How can we verify that an LLM’s reasoning is accurately grounded in knowledge? To address this question, we introduce a novel evaluation suite that systematically assesses the knowledge grounding of intermediate reasoning. Our framework comprises three key components. (1) Principal Knowledge Collection , a large-scale repository of atomic knowledge essential for reasoning. Based on the collection, we propose (2) knowledge-grounded evaluation metrics designed to measure how well models recall and apply prerequisite knowledge in reasoning. These metrics are computed by our (3) evaluator LLM, a lightweight model optimized for cost-effective and reliable metric computation. Our evaluation suite demonstrates remarkable effectiveness in identifying missing or misapplied knowledge elements, providing crucial insights for uncovering fundamental reasoning deficiencies in LLMs. Beyond evaluation, we demonstrate how these metrics can be integrated into preference optimization, showcasing further applications of knowledge-grounded evaluation. Our evaluation suite is publicly available. 1 1 1 https://github.com/dmis-lab/PKCollection . Figure 1: Example of a reasoning failure. The model correctly applies the knowledge of “ combining fractions ” but omits prerequisite steps such as “ simplifying a fraction ,” which leads to an incorrect answer. This underscores the importance of evaluating reasoning steps through knowledge-grounded assessments. 1 Introduction Recently, large language models (LLMs) OpenAI 2024b ; DeepSeek-AI et al. 2025 have demonstrated remarkable reasoning capabilities. These models have achieved substantial performance gains by generating their own intermediate reasoning steps before arriving at a final answer. However, such gains are often assessed solely through end-task metrics, which evaluates only the final output. While this simplicity offers practical efficiency in evaluation, it provides limited insights into how reliably or meaningfully the model performs reasoning, failing to detect errors in intermediate steps. To address this, prior works ( Prasad et al. 2023 ; Golovneva et al. 2023 ; Chen et al. 2023 ; Hao et al. 2024 ) have focused on evaluating the validity of the intermediate reasoning steps that lead to the final output. They evaluate the reasoning steps by identifying errors like hallucinations Huang et al. 2025 and logical flaws. Yet, these approaches do not adequately measure how a model utilizes the essential knowledge required for problem-solving, making it difficult to provide interpretable feedback on model behavior. Figure 1 exemplifies this limitation, showing how a model can apply partial knowledge correctly while overlooking other prerequisite steps. While it correctly applies combining fractions, it neglects necessary knowledge, such as simplifying a fraction and converting the result into a mixed number. To systematically assess how precisely a model utilizes essential knowledge, we propose an evaluation suite consisting of the following three components: We first construct (1) Principal Knowledge Collection ( PK Collection ), a large-scale resource of atomic knowledge essential for solving the target tasks. This knowledge is collected from multiple top-performing LLMs, clustered into coherent chunks, and eventually refined into principal knowledge for each chunk. In particular, we built this collection based on the MMLU benchmark Hendrycks et al. 2020 , resulting in 112k principal knowledge units (PK units). Grounded in the PK Collection , we introduce (2) novel evaluation metrics, including knowledge-grounded precision, recall, and F1 score. The LLM-based evaluation process involves identifying the utilization of PK units within the predicted rationale. These metrics measure how precisely the model applies the knowledge and whether it recalls the necessary information accurately. Furthermore, using proprietary LLMs for reasoning evaluation is costly and impractical at scale. To overcome this, we develop (3) a light-weight LLM evaluator distilled from a proprietary teacher LLM, achieving high agreement with reduced computational overhead. We also validate PK Collection for factuality and relevance, confirming that it maintains high quality in both aspects. To additionally explore the potential applications of our evaluation suite, we investigate its use in controlling reasoning conciseness with recent preference optimization techniques. Following the approach of Pang et al. 2025 , we adopt a framework that applies Direct Preference Optimization (DPO) Rafailov et al. 2023 . By selecting preferred samples based on our evaluation metrics, we observe that models could efficiently generate reasoning steps while maintaining end performance. This result demonstrates that aligning LLM with our evaluation metrics effectively guides it to concisely reach the final result for solving the task. Our experimental results and analysis demonstrate that our knowledge-grounded metrics effectively evaluate knowledge utilization and provide interpretable feedback on model behavior. Moreover, reasoning preference optimization guided by our knowledge-grounded metric not only boosts performance but also enables control over the model’s knowledge usage and token consumption. Our contributions are summarized in threefold: • We construct Principal Knowledge Collection , a large-scale resource containing 112K principal knowledge for solving questions from diverse domains with high accuracy and relevance. • We propose a novel evaluation system equipped with knowledge-grounded metrics and a distilled evaluator LLM, which assesses the utilization of principal knowledge in LLM reasoning. • We extend our evaluation suite to enable more controllable model reasoning via preference optimization. Our analysis showcases how this approach encourages more concise reasoning, which also leads to reduced token consumption. 2 Related Works Figure 2: Overview of our evaluation suite for assessing LLM reasoning steps via knowledge grounding. We first construct the Principal Knowledge Collection (§ 3.1 ) through a two-step process: Given a task, we (a) collect atomic knowledge crucial for task resolution from multiple top-performing LLMs. Subsequently, we (b) cluster these units into semantically coherent groups to obtain the principal knowledge for each cluster. Grounded on this collection, we (c) evaluate the model’s reasoning steps (§ 3.2 ) by measuring whether the model accurately retrieves and applies the principal knowledge using our proposed knowledge recall and precision metrics. LLM Reasoning Steps Chain-of-Thought ( Wei et al. 2022 ) encourages the generation of intermediate reasoning steps, improving LLMs’ ability to solve complex tasks. Self-consistency ( Wang et al. 2023 ) improves reliability by sampling multiple reasoning paths and selecting the most frequent answer through majority voting. Tree-of-Thought ( Yao et al. 2023 ) employs heuristic-based search strategies, such as depth-first and breadth-first search, to explore more effective reasoning paths. Chain-of-Knowledge ( Li et al. 2024 ) focuses on improving factuality by dynamically grounding reasoning steps with external sources. LLM-based Evaluation Recent studies explore the use of LLMs as evaluators, automating response assessment, and reducing the dependence on human annotations ( Zheng et al. 2023 ; Fu et al. 2023 ; Lyu et al. 2024 ) , and also show strong correlation with human judgements ( Liusie et al. 2024 ) . To mitigate dependence on proprietary language models, subsequent research has explored training models specifically designed for evaluation tasks ( Li et al. 2023 ; Kim et al. 2024a ; Kim et al. 2024b ) . LLM Reasoning Evaluation Recent advancements have proposed fine-grained methods for evaluating intermediate reasoning in LLMs. Beyond assessing sentence-level correctness, prior work has examined the alignment of intermediate steps with the final answer