Learning When Not to Decide: A Framework for Overcoming Factual Presumptuousness in AI Adjudication DOI: XXXXXXX.XXXXXXX Conference: Twenty-First International Conference on Artificial Intelligence and Law; June 08–12, 2026; Singapore, Singapore CCS: Computing methodologies Natural language processing CCS: Applied computing Law

Mohamed Afane, Emily Robitschek, Derek Ouyang, Daniel E. Ho · 2026

Read Learning When Not to Decide: A Framework for Overcoming Factual Presumptuousness in AI Adjudication DOI: XXXXXXX.XXXXXXX Conference: Twenty-First International Conference on Artificial Intelligence and Law; June 08–12, 2026; Singapore, Singapore CCS: Computing methodologies Natural language processing CCS: Applied computing Law by Mohamed Afane, Emily Robitschek, Derek Ouyang, Daniel E. Ho (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-19895.txt.

Introduction

Afane and three colleagues tackle AI overconfidence in unemployment insurance rulings, where judges must first decide if the facts suffice. Standard retrieval tools scored only 15 percent when facts fell short, while stronger prompts refused even clear cases. Their SPEC method forces a written list of missing facts before any ruling. It reached 89 percent overall and held back correctly on thin cases, showing that this flaw is systematic yet fixable.

Catalog record

CREATORMohamed Afane, Emily Robitschek, Derek Ouyang, Daniel E. Ho
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49122
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.19895

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Learning When Not to Decide: A Framework for Overcoming Factual Presumptuousness in AI Adjudication DOI: XXXXXXX.XXXXXXX Conference: Twenty-First International Conference on Artificial Intelligence and Law; June 08–12, 2026; Singapore, Singapore CCS: Computing methodologies Natural language processing CCS: Applied computing Law Mohamed Afane, Emily Robitschek, Derek Ouyang, Daniel E. Ho Abstract . A well-known limitation of AI systems is presumptuousness : the tendency of AI systems to provide confident answers when information may be lacking. This challenge is particularly acute in legal applications, where a core task for attorneys, judges, and administrators is to determine whether evidence is sufficient to reach a conclusion. We study this problem in the important setting of unemployment insurance adjudication, which has seen rapid integration of AI systems and where the question of additional fact-finding poses the most significant bottleneck for a system that affects millions of applicants annually. First, through a collaboration with the Colorado Department of Labor and Employment, we secure rare access to official training materials and guidance to design a novel benchmark that systematically varies in information completeness. Second, we evaluate four leading AI platforms and show that standard RAG-based approaches achieve an average of only 15% accuracy when information is insufficient. Third, advanced prompting methods improve accuracy on inconclusive cases but over-correct, withholding decisions even on clear cases. Fourth, we introduce a structured framework requiring explicit identification of missing information before any determination (SPEC, S tructured P rompting for E vidence C hecklists). SPEC achieves 89% overall accuracy, while appropriately deferring when evidence is insufficient—demonstrating that presumptuousness in legal AI is systematic but addressable, and that doing so is a necessary step towards systems that reliably support, rather than supplant, human judgment wherever decisions must await sufficient evidence. 1. Introduction Legal reasoning has always required navigating uncertainty. Statutes employ vague standards—“reasonable efforts,” “good cause,” “willful misconduct”—that demand interpretation against specific facts. Evidence is incomplete; witnesses contradict each other; circumstances resist clean categorization. For human adjudicators, this is familiar terrain. For artificial intelligence (AI), it has proven treacherous. The earliest legal expert systems, developed in the 1980s and 1990s, encoded statutory rules as deterministic decision trees. They worked well when law reduced to bright-line tests—twenty weeks of employment, a specific dollar threshold—but collapsed when confronting the uncertainty endemic to legal reasoning. Governing law proved too complex to represent directly as symbolic logic; as Kevin Ashley documented, they “did not possess the capacity to represent uncertainty or engage in non-monotonic reasoning about changing facts” ( Ashley 2017 ) . Modern transformer-based architectures have not resolved this limitation. Despite remarkable linguistic capabilities, large language models (LLM) generate fluent, assured outputs regardless of whether the underlying evidence warrants such confidence ( Ye et al. 2024 ) . The systems that once failed because they could not handle uncertainty now fail because they cannot recognize insufficiency. This presumptuousness is a general problem for AI deployed in reasoning tasks, but it is particularly acute in law—and nowhere more so than in administrative adjudication, where AI deployment is accelerating, and errors directly harm individuals ( Engstrom and Ho 2020 ) . Unemployment insurance (UI) offers a compelling domain to study this problem. The governing law is formidably complex: scattered statutory provisions, vague eligibility standards, and jurisdiction-specific exceptions that resist systematic codification ( Pahlka 2023 ; Young et al. 2022 ) . Developing sound adjudicative judgment can take years. When a researcher asked a California UI claims processor why he kept deferring questions to senior colleagues, he replied: “I’ve only been here 17 years. The folks who really know how UI works have been here for 25 years or more” ( Pahlka 2023 ) . Agencies maintain extensive training programs to cultivate this judgment in new adjudicators—and increasingly look to AI to help. The pressure to automate is intense. During the pandemic, unemployment systems nationwide collapsed under unprecedented claim volumes, not primarily because they lacked processing capacity, but because fact-finding became the bottleneck. Determining eligibility requires gathering and weighing evidence: why did the claimant leave their job? Did the employer provide adequate documentation? Is the claimant’s account consistent with the record? When systems could not perform this evidentiary work adequately, they failed ( Pahlka 2023 ) . Agencies find themselves caught between competing demands: rely on merit staff and fail to meet timeliness requirements, or automate and risk running afoul of rules designed to ensure human judgment ( Daniel E. Ho et al. 2025 ) . The Colorado Department of Labor and Employment (CDLE) has piloted AI tools designed to help adjudicators “do the work more accurately and efficiently” rather than replace human judgment ( Alms 2024 ) . New Hampshire is testing an adjudication assistant that collects more complete information at the start of claims ( Sta 2023 ) . The question is not whether AI will be integrated into benefits administration—it already is—but whether these systems can recognize the limits of their own knowledge. The consequences of getting this wrong are severe. In 2015, Michigan’s automated UI system falsely accused over 20,000 residents of fraud, issuing confident determinations without adequate evidentiary support. The system’s designers had not equipped it to recognize when information was insufficient; it simply decided. Many residents had wages garnished or tax refunds seized before the errors came to light. At least one person took her own life after being assessed $50,000 in penalties ( Felton 2016 ) . The pattern extends to eligibility determinations: AI systems that deny without sufficient evidence strand workers who may qualify; systems that approve without sufficient evidence create overpayments that agencies later claw back. Either way, the harm flows from deciding before the facts warrant it. A common approach to addressing these concerns is to have a human in the loop. But this framing obscures critical design choices. Two well-known failure modes in human–AI interaction are automation bias, where humans over-rely on algorithmic suggestions, and algorithm aversion, where humans dismiss useful recommendations ( Alon-Barkat and Busuioc 2023 ) . Humans can identify erroneous algorithmic outputs and override them, but only when systems provide clear signals about uncertainty rather than overconfident predictions that humans must second-guess ( De-Arteaga et al. 2020 ) . A system that explicitly identifies missing information before reaching a determination provides exactly such a signal. Most benchmarks for evaluating AI systems, including those in legal domains, sidestep this question entirely. They assess performance when all relevant information is provided, testing whether models reach correct conclusions, not whether they can recognize when conclusions are unwarranted ( Guha et al. 2023 ; Hendrycks et al. [n. d.] ; Holzenberger et al. 2020 ; Zheng et al. 2021 ; Koreeda and Manning 2021 ) . This paper addresses that gap. Through a collaboration with CDLE, we obtain rare access to official training materials—the actual guidance used to teach human adjudicators when they have enough information to decide. We use these materials, alongside governing statutes provided through retrieval-augmented generation (RAG), to construct eligibility scenarios that vary systematically in information completeness: cases that are clearly eligible, clearly ineligible, and cases where a competent adjudicator should recognize that determination must await further fact-finding. We contribute: (1) The first legal adjudication dataset that precisely controls the presence or absence of legally operative

Read the complete text