Before you <think> , monitor: Implementing Flavell’s metacognitive framework in LLMs

Nick Oh · 2025

Read Before you <think> , monitor: Implementing Flavell’s metacognitive framework in LLMs by Nick Oh (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-16374.txt.

Introduction

Nick Oh, writing alone, gives language models a dose of metacognition, the habit of thinking about their own thinking. Current fixes split two ways: plan-first methods never check whether the plan worked, and check-later methods start writing blind. Oh merges them using Flavell's 1979 model of cognitive monitoring, in a loop of monitor, generate, and verify.

On GSM8K, a set of grade-school math word problems, the loop answers about seventy-five percent correctly. That beats two refinement baselines stuck near sixty-seven to sixty-eight percent. It also needs fewer retries, about one point three attempts against two, though each run costs roughly a third more compute. Early days yet: the test covers only arithmetic, and broader trials must show whether upfront monitoring pays off elsewhere.

Catalog record

CREATORNick Oh
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE27 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-16374

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Before you <think> , monitor: Implementing Flavell’s metacognitive framework in LLMs Nick Oh Abstract Current approaches to enhancing LLM reasoning follows two isolated paradigms: Monitor-Generate methods like Plan-and-Solve ( Wang et al. 2023 ) and SELF-DISCOVER ( Zhou et al. 2024 ) excel at strategic planning but lack mechanisms to verify whether selected strategies succeed; while Generate-Verify approaches like Self-Verification ( Weng et al. 2022 ) and SELF-REFINE ( Madaan et al. 2023 ) iteratively refine outputs but commence generation blindly without task assessment. This separation creates inefficiencies – strategies fail without feedback, and refinement occurs without strategic grounding. We address this gap by implementing Flavell 1979 ’s cognitive monitoring model ( Flavell 1979 ) from the broader Monitor-Generate-Verify framework ( Oh & Gobet 2025 ) , operationalising it as a three-phase iterative system. On GSM8K, preliminary results show 75.42% accuracy versus 68.44% for SELF-REFINE and 67.07% for Self-Verification, while requiring fewer attempts (1.3 vs 2.0) at 27-37% increased inference cost. These initial findings suggest upfront monitoring produces higher-quality initial solutions that reduce refinement needs, though evaluation beyond arithmetic reasoning is needed to establish generalisability. 1 Introduction This preliminary experimental report documents our initial implementation and testing of Flavell 1979 ’s ( Flavell 1979 ) cognitive monitoring model for enhancing LLM. We operationalise the Monitor-Generate-Verify framework proposed by Oh & Gobet 2025 , creating a system where models explicitly assess task difficulty before attempting solutions, adapt their computational resources accordingly, and evaluate outputs along structured metacognitive dimensions. Our experiments on a subset of GSM8K problems compare this approach against Self-Verification and SELF-REFINE baselines using Llama-3.1-8B-Instruct . While our evaluation is limited to 659 arithmetic problems with a single model, the results suggest that explicit monitoring may reduce the need for extensive iterative refinement by producing better initial solution attempts. We present these findings as an early exploration of whether cognitive science principles, particularly metacognitive monitoring, can inform more effective reasoning architectures for language models. This paper is structured as follows: Section 2 situates our approach within existing paradigms. Section 3 presents our implementation of Flavell’s cognitive monitoring model. Section 4 evaluates performance against baselines. Section 5 discusses limitations and future directions. Section 6 reflects on the methodological implications of our approach. 2 Related Works 2.1 Monitor-Generate (MG) Methods Monitor-Generate (MG) approaches recognise that effective reasoning requires understanding task structure before attempting solutions. Zero-shot prompting methods have evolved from simple planning to sophisticated strategy discovery: Plan-and-Solve ( Wang et al. 2023 ) replaces generic “think step by step” with structured guidance for problem decomposition and careful execution, achieving 76.7% accuracy on arithmetic tasks; SELF-DISCOVER ( Zhou et al. 2024 ) enables LLMs to identify task-intrinsic reasoning structures through meta-prompts (SELECT, ADAPT, IMPLEMENT), yielding 27-32% improvements on reasoning tasks; and Meta-Reasoning Prompting ( Gao et al. 2024 ) functions as an adaptive router, evaluating multiple reasoning methods against task requirements to select the optimal approach. Few-shot prompting methods leverage exemplars for enhanced strategy selection, with Strategic Chain-of-Thought ( Wang et al. 2024 ) first eliciting strategic knowledge then applying it systematically (achieving 21.05% improvement on GSM8K), and HYBRIDMIND ( Han et al. 2024 ) employing a meta-selector to choose between natural language, symbolic, or hybrid reasoning. Reinforcement learning based approaches like Elastic Reasoning (Scalable Chain-of-Thought) ( Xu et al. 2025 ) optimise strategy through trained budget allocation between thinking and solution phases. Despite their sophistication in pre-generation planning, these methods lack mechanisms to verify whether selected strategies actually succeed or to learn from failed attempts. 2.2 Generate-Verify (GV) Methods Generate-Verify (GV) approaches focus on iterative refinement through self-evaluation. Self-Verification ( Weng et al. 2022 ) generates multiple candidates then validates through backward reasoning, systematically masking conditions to reconstruct original problems and selecting based on aggregated verification scores. SELF-REFINE ( Madaan et al. 2023 ) implements a three-phase cycle where the same model serves as generator, critic, and refiner, maintaining complete iteration history to prevent repeating mistakes, achieving 20% average improvement without additional training. Recent advances emphasise sophisticated verification mechanisms: ReVISE ( Lee et al. 2025 ) enables intrinsic self-verification through structured curriculum learning with confidence-aware decoding, while theoretical work formalises self-improvement through the ‘generation-verification gap’ that scales with model pre-training FLOPs ( Song et al. 2024 ) . The pivotal role of verifier quality has been demonstrated by Zhang et al. 2024 , showing substantial gains with strong verifiers but limitations with weak self-verifiers. However, even strong verification cannot fully overcome the ‘prefix dominance trap’ ( Luo et al. 2025 ) : once models begin with suboptimal reasoning strategies, performance degrades by nearly 20% with minimal recovery through subsequent verification. This exposes a fundamental limitation – these approaches commence generation without assessing task characteristics or retrieving problem-solving strategies, leading to inefficient exploration that initial strategic planning could avoid. 3 Monitor-Generate-Verify Framework The preceding analysis reveals a fundamental gap. MG methods excel at strategic planning but cannot verify success, while GV approaches refine outputs iteratively but lack initial task assessment. Addressing these complementary limitations, we implement the Monitor-Generate-Verify (MGV) framework ( Oh & Gobet 2025 ) – specifically operationalising Flavell 1979 ’s cognitive monitoring model ( Flavell 1979 ) as a three-phase iterative reasoning system. Rather than operating within a single paradigm, MGV synthesises both: it monitors task characteristics to inform strategy selection (addressing GV’s blind generation), executes with adaptive parameters, then verifies outcomes to guide subsequent cycles (addressing MG’s lack of systematic validation). This creates a complete metacognitive loop where verification informs monitoring, monitoring guides generation, and generation produces solutions whose evaluation feeds back to refine initial assessments. The framework operates through T cycles, where each cycle \tau\in\{0,1,...,T-1\} consists of three phases. Our implementation employs zero-shot prompting throughout, avoiding the need for task-specific examples while testing the generality of metacognitive principles (complete prompts are provided in Appendix A ). Following Flavell’s assumption of pre-existing metacognitive knowledge, we adopt the 20 problem-solving strategies compiled by Didolkar et al. 2024 , who instructed LLMs to assign skill labels to GSM8K problems. While this knowledge base was constructed using gpt-4-0613 , raising questions about cross-model transferability – akin to humans having privileged access to their own metacognition ( Nelson & Narens 1990 ) and findings that LLMs better predict their own behaviour than other models’ ( Binder et al. 2024 ) – we adopt it as a reasonable starting point for this preliminary investigation. This knowledge base primarily represents \mathcal{MK}_{\text{Strategy}} from Flavell’s three categories ( \mathcal{MK}_{\text{Agent}} , \mathcal{MK}_{\text{Task}} , \mathcal{MK}_{\text{Strategy}} ), focusing on problem-solving approaches rather than agent capabilities or task characteristics. Algorithm 1 Flavell’s Model of Cognitive Monitoring 1: task \mathcal{T} , model M , strategies \mathcal{MK} , prompts \mathcal{P}=\{p_{monitor},p_{strategy},p_{execute},p_{verify}\} 2: Initialize \mathcal{S}_{0}=\text{ACTIVE} , \tau=0 , \mathcal{ME}_{\text{evaluative}}^{-1}=\emptyset 3: while \mathcal{S}_{\tau}=\text{ACTIVE} and \tau<T do 4: // Monitor: Assess task difficulty 5: \mathcal{ME}_{\text{difficulty}}^{\tau},features_{\tau}=M(p_{mon}\|\mathcal{T}\|\mathcal{ME}_{\text{evaluative}}^{\tau-1}) \triangleright Assess task difficulty 6: 7: // Generate: Apply cognitive strategy 8: strategy_{\tau}=M(p_{str}\|features_{\tau}\|\mathcal{ME}_{\text{difficulty}}^{\tau}\|\mathcal{MK}) \triangleright Choose approach 9: solution_{\tau}=M(p_{exe}\|\mathcal{T}\|strategy_{\tau}) \triangleright Execute strategy with adaptive params 10: 11: // Verify: Evaluate performance 12: \mathcal{ME}_{\text{evaluative}}^{\tau}=M(p_{ver}\|\mathcal{T}\|solution_{\tau}) \triangleright Evaluate output quality 13: 14: \mathcal{S}_{\tau+1}=\textbf{if }\text{mean}(\mathcal{ME}_{\text{evaluative}}^{\tau})\geq 0.85\textbf{ then }\text{TERMINATE }\textbf{else }\text{ACTIVE} 15: \tau=\tau+1 16: end while 17: return y_{\arg\max_{i}\text{mean}(\mathcal{ME}_{\text{evaluative}}^{i})} The following subsections detail each component of our implementation, examining how monitoring assess task difficulty, how generation employs dual-stage processing,

Read the complete text