Zero-Shot Verification-guided Chain of Thoughts

Jishnu Ray Chowdhury, Cornelia Caragea · 2025

Read Zero-Shot Verification-guided Chain of Thoughts by Jishnu Ray Chowdhury, Cornelia Caragea (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-13122.txt.

Introduction

Jishnu Ray Chowdhury and Cornelia Caragea let a language model check its own reasoning steps, with no examples and no trained checker. Step by step prompting and separate verifiers both help models reason, but verifiers are usually fine tuned or fed hand written examples. Their 2025 paper stays fully zero shot.

A new prompt, COT STEP, helps the model split its reasoning into clean steps. Two further prompts turn the same kind of model into a verifier that scores each chain's correctness. The authors test how well these verifiers sort good chains from bad, and try several ways to steer reasoning with the scores. Tasks span math and commonsense problems across different models. The study maps what self verification can do unaided.

Catalog record

CREATORJishnu Ray Chowdhury, Cornelia Caragea
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-13122

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Zero-Shot Verification-guided Chain of Thoughts Jishnu Ray Chowdhury, Cornelia Caragea Abstract Previous works have demonstrated the effectiveness of Chain-of-Thought (COT) prompts and verifiers in guiding Large Language Models (LLMs) through the space of reasoning. However, most such studies either use a fine-tuned verifier or rely on manually handcrafted few-shot examples. In contrast, in this paper, we focus on LLM-based self-verification of self-generated reasoning steps via COT prompts in a completely zero-shot regime . To explore this setting, we design a new zero-shot prompt, which we call COT STEP, to aid zero-shot decomposition of reasoning steps and design two new zero-shot prompts for LLM-based verifiers. We evaluate the verifiers’ ability to classify the correctness of reasoning chains and explore different ways to use verifier scores in guiding reasoning for various mathematical and commonsense reasoning tasks with different LLMs. 1 Introduction Large Language Models (LLMs) Dubey et al. 2024 ; Brown et al. 2020 have revolutionized the field of NLP by enabling state-of-the-art (SOTA) performance in several tasks merely by smart prompting Wei et al. 2022a ; Bubeck et al. 2023 . A critical landmark in the art of prompting for multi-step reasoning tasks is the Chain-of-Thought (COT) prompting Nye et al. 2022 ; Wei et al. 2022b , which elicits LLMs to generate step-by-step reasoning chains before providing the answer to a given question. Building upon COT prompting, several recent works incorporate a verifier mechanism to improve LLMs’ performance. For instance, Cobbe et al. 2021 ; Li et al. 2023 ; Weng et al. 2023 use a verifier to evaluate the correctness and score each reasoning step, whereas Gandhi et al. 2023 ; Yao et al. 2023 ; Hao et al. 2023 ; Hao et al. 2024 use a verifier’s scores to do a tree-search over the space of reasoning steps. However, these works have several drawbacks including: 1. most of them rely on few-shot prompting, i.e., they rely on exemplars (often manually handcrafted) of how the task is to be performed. This selection of the exemplars brings additional challenges as to which exemplars should be selected, how many, etc; 2. The verifier is also typically either fine-tuned on labeled datasets or rely on further verification-related few-shot exemplars. Properly fine-tuning a verifier may require a large number of human annotations which may be impractical in different tasks and domains. In contrast, in this paper, we focus on LLM-based self-verification of self-generated reasoning steps via COT prompts in a completely zero-shot regime . To achieve this, we propose a new zero-shot COT-based prompt, called COT STEP, for generating reasoning steps (i.e., to aid zero-shot decomposition of reasoning steps) and two new zero-shot verification prompts for using LLMs as verifiers to verify their own reasoning steps (i.e., evaluate whether the steps are correct or not). The zero-shot regime is valuable because in this setup, no manually selected or handcrafted examples are required. Moreover, in the complete zero-shot setting that we explore, there is no reliance on any additional data resource for exemplar construction. Furthermore, a general purpose zero-shot prompt can be used to set up a convenient model interface for users to request different kinds of reasoning tasks without having to worry about providing task-specific exemplars. In exploring the zero-shot regime, we ask three research questions (RQ1-3) in this paper. RQ1: How do different zero-shot COT prompts compare against each other across different LLMs? RQ2: How well do LLM-based verifiers perform in evaluating the correctness of reasoning steps? RQ3: How well can LLM-based verifier scores be used in selecting the right answer among many or in guiding reasoning in real-time? We address each of the questions in turn - RQ1 in \lx@sectionsign 3 , RQ2 in \lx@sectionsign 4 , and RQ3 in \lx@sectionsign 5 and evaluate them on various mathematical and commonsense reasoning tasks with different LLMs. While investigating RQ1, we find that our COT STEP prompt keep up competitively with other prompts alongside enabling automatic step decomposition. While investigating RQ2, we find that a zero-shot COT-variant of verification prompt tends to perform reasonably in verifying the correctness of mathematical reasoning steps. While investigating RQ3, we do find a noticeable benefit from using the verifier score in a step-level greedy search but the benefit vanishes when self-consistency Wang et al. 2023b is used. 2 Main Components In this section, we discuss the main components of our experiments that are relevant for all our research questions (RQ1-3). 2.1 Models For our experiments, we primarily focus on two openly-licensed models in the main paper - (1) SOLAR (SOLAR 10.7B Instruct) 1 1 1 SOLAR is an instruction-tuned depth-scaled variation of Llama 2. Kim et al. 2024 and (2) Phi3 (Phi 3 Mini 128K Instruct) Abdin et al. 2024 and show additional experiments on other models such as Mistral (Mistral 7B Instruct v0.2) Jiang et al. 2023 and Llama3 (LLama 3.0 8B Instruct) Dubey et al. 2024 in Appendix B . Prompt Style GSM8K GSMH AQuA StratQA CSQA GSM8K GSMH AQuA StratQA CSQA SOLAR 10.7B Instruct SOLAR 10.7B Instruct + SC@20 Base 17.06 5.38 6.69 31.09 63.96 17.67 7.28 30.71 65.42 72.40 COT 73.62 28.96 6.69 58.47 66.67 83.40 37.38 42.9 73.10 78.79 PS+ 68.39 26.61 10.63 30.87 51.27 84.9 39.20 47.24 69.78 80.43 TAB COT 65.2 26.69 8.66 52.66 65.19 80.52 36.16 41.73 73.89 80.51 COT STEP 73.54 30.63 11.02 58.95 67.32 84.91 37.45 38.58 73.76 79.85 Phi3 Mini 128K Instruct Phi3 Mini 128K Instruct + SC@20 Base 51.40 24.19 19.29 9.2 69.53 77.64 38.29 45.67 40.13 73.55 COT 84.84 37.6 18.89 44.63 69.37 91.28 45.03 64.57 61.75 76.33 PS+ 81.5 36.24 14.96 17.21 56.76 90.9 46.02 66.54 55.72 75.18 TAB COT 73.77 32.98 12.6 45.98 67.90 86.81 44.05 57.87 63.67 74.86 COT STEP 83.25 38.59 15.75 51.0 71.75 90.75 45.49 65.75 63.49 77.07 Table 1: Comparison of accuracy from different prompting strategies across different datasets. GSMH represents GSM-HARD, StratQA represents StrategyQA, and CSQA represents CommonsenseQA. SC@20 represents Self-consistency with majority voting of 20 sample generations. We bold the best result per block per dataset. 2.2 Existing Prompts In our experiments, we consider four major existing zero-shot prompt strategies. We describe them here. Note that we use a simplified representation of user-assistant conversational turns to describe the prompt structure for generality and brevity. 2 2 2 Different models based on how they were instruction-tuned, require different kinds of templates to represent user-assistant conversational turns which we use accordingly in experiments. The specific template information can be found in Huggingface Wolf et al. 2020 tokenizer configs in the corresponding model pages. (P1) Base: Base represents the baseline zero-shot prompting style for question answering that we use. Given a question, such as “What is 2+2?", the prompt to elicit an answer would look like: “User: Q: What is 2+2? \n Assistant: A:" . (P2) COT: COT (Chain Of Thoughts) represents the standard zero-shot COT prompt Kojima et al. 2022 . This can be expressed as “User: Q: What is 2+2? \n Assistant: Let’s think step by step." . Such a prompt is designed to prime the models to generate more explicit chain of reasoning steps before producing an answer. (P3) PS+: PS+ represents the plan-and-solve zero shot PS+ prompt presented by Wang et al. 2023a as an

Read the complete text