An Industrial-Scale Insurance LLM Achieving Verifiable Domain Mastery and Hallucination Control without Competence Trade-offs

Qian Zhu, Xinnan Guo, Jingjing Huo, Jun Li, Pan Liu, Wenyan Yang, Wanqing Xu, Xuan Lin · 2026

Read An Industrial-Scale Insurance LLM Achieving Verifiable Domain Mastery and Hallucination Control without Competence Trade-offs by Qian Zhu, Xinnan Guo, Jingjing Huo, Jun Li, Pan Liu, Wenyan Yang, Wanqing Xu, Xuan Lin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-14463.txt.

Introduction

Zhu and co-authors build INS-S1, a language model family made for insurance work. Insurance tasks demand strict rule following with no room for invented facts. Prior fixes often trade general skill for domain skill. Their training pairs a verifiable data synthesis system with a staged curriculum. The curriculum mixes verified reasoning rewards with AI feedback. They also release INSEva, a test set with over 39,000 samples. INS-S1 beats DeepSeek-R1 and Gemini-2.5-Pro on domain tasks, the authors report, while keeping general skill.

Catalog record

CREATORQian Zhu, Xinnan Guo, Jingjing Huo, Jun Li, Pan Liu, Wenyan Yang, Wanqing Xu, Xuan Lin
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE42909
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.14463

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

An Industrial-Scale Insurance LLM Achieving Verifiable Domain Mastery and Hallucination Control without Competence Trade-offs Qian Zhu, Xinnan Guo, Jingjing Huo, Jun Li, Pan Liu, Wenyan Yang, Wanqing Xu, Xuan Lin Abstract Adapting Large Language Models (LLMs) to high-stakes vertical domains like insurance presents a significant challenge: scenarios demand strict adherence to complex regulations and business logic with zero tolerance for hallucinations. Existing approaches often suffer from a Competency Trade-off—sacrificing general intelligence for domain expertise—or rely heavily on RAG without intrinsic reasoning. To bridge this gap, we present INS-S1 , an insurance-specific LLM family trained via a novel end-to-end alignment paradigm. Our approach features two methodological innovations: (1) A Verifiable Data Synthesis System that constructs hierarchical datasets for actuarial reasoning and compliance; and (2) A Progressive SFT-RL Curriculum Framework that integrates dynamic data annealing with a synergistic mix of Verified Reasoning (RLVR) and AI Feedback (RLAIF). By optimizing data ratios and reward signals, this framework enforces domain constraints while preventing catastrophic forgetting. Additionally, we release INSEva , the most comprehensive insurance benchmark to date (39k+ samples). Extensive experiments show that INS-S1 achieves SOTA performance on domain tasks, significantly outperforming DeepSeek-R1 and Gemini-2.5-Pro. Crucially, it maintains top-tier general capabilities and achieves a record-low 0.6% hallucination rate (HHEM). Our results demonstrate that rigorous domain specialization can be achieved without compromising general intelligence. 0 0 footnotetext: {zq371417, daxuan.lx}@antgroup.com 1 Introduction Figure 1: Performance comparison of INS-S1-235B against SOTA models. Recent LLMs (e.g., DeepSeek-R1 ( DeepSeek-AI et al. 2025 ) , Gemini ( Comanici et al. 2025 ) , Qwen3 ( Yang et al. 2025 ) ) have established new standards in reasoning. While domain models have advanced in finance and medicine, the insurance sector remains underexplored . Adapting to this domain demands strict compliance and a robust mechanism against hallucinations. Current solutions face three limitations: (1) Competency Trade-off and Shallow Adaptation: Existing vertical models often sacrifice general intelligence for domain expertise. For instance, financial models like DianJin-R1 ( Zhu et al. 2025 ) suffer from severe catastrophic forgetting ( Kirkpatrick et al. 2017 ) , struggling to balance specialized knowledge with general reasoning. Most adaptations rely on superficial RAG ( Lewis et al. 2020 ) or naive fine-tuning, failing to internalize the multi-step reasoning required for actuarial pricing, resulting in models that lack deep reasoning capabilities. (2) High-Stakes Hallucination Risks: Insurance decisions demand auditable reasoning paths. Current models often operate as “black boxes,” hallucinating policy terms or fictional cases. In a regulated industry, such errors lead not just to user distrust but to severe legal liabilities. (3) Lack of Standardized Evaluation: The absence of a comprehensive, specialized benchmark hinders the iterative optimization of insurance LLMs. General financial benchmarks ( Guo et al. 2025 ) fail to capture the nuances of claim adjudication, underwriting logic, and regulatory compliance. To address these challenges, we present INS-S1 , a family of scalable insurance LLMs. Our approach integrates three core innovations: • Verifiable Insurance Data Synthesis System: Transcending simple knowledge injection, we design a hierarchical dataset construction pipeline. It features fine-grained sub-task decomposition and a multi-source synthesis framework that enforces strict logic constraints, ensuring the model masters complex actuarial reasoning while maintaining general intelligence via self-distillation. • Progressive SFT-RL Curriculum Framework: We propose a two-stage training paradigm that integrates Dynamic Data Annealing in SFT with a synergistic RL framework (RLVR ( Lightman et al. 2023 ) and RLAIF ( Lee et al. 2023 ) ). By optimizing data ratios and leveraging GRPO ( Shao et al. 2024 ) /GSPO ( Zheng et al. 2025 ) optimization, this framework enables the model to master complex actuarial reasoning while strictly aligning outputs with insurance requirements. • INSEva Benchmark: We release the industry’s most comprehensive evaluation benchmark, comprising 39k samples across 9 core dimensions and 50+ task types. Utilizing an automated “LLM-as-a-Judge” pipeline, INSEva provides a robust standard for assessing domain expertise. Consequently, as shown in Figure , INS-S1-235B bridges the domain gap, scoring 90.14 on INSEva, surpassing general baselines like DeepSeek-R1 (82.47) by a significant margin. Crucially, it exhibits negligible degradation on general benchmarks compared to the backbone, while our RL alignment successfully constrains the hallucination rate to 0.6% . The detailed architecture is illustrated in Appendix . 2 Data 2.1 Overview We implement a capability-driven data strategy centered on three pillars. (1) Insurance Domain Mastery: We construct compliant corpora using dual synthesis pipelines (No-Query and With-Query) to optimize domain task quality. (2) Effective Hallucination Control: We prioritize faithfulness via four atomic tasks and RAG adaptation, balancing rigor with expression. (3) General Capability Preservation: To prevent catastrophic forgetting, we curate a High-Fidelity Chinese dataset (spanning STEM, Math, Exams) and employ Self-Distillation to align general data with the base model’s style. Figure 2: Data Synthesis Diagram. 2.2 Data Construction Details 2.2.1 Insurance-Specific Data Construction Knowledge Injection Tasks (No-Query). We process multi-modal corpora (textbooks, regulations, tables) via a pipeline: Multi-modal Parsing \to Markdown Representation \to QA Generation. To prevent reasoning over-reliance for rote knowledge, we employ Instruction Isolation with “Empty Think Tags” ( <think></think> ), which improved accuracy on theory benchmarks by 3.11% compared to baselines. Cognitive Alignment Tasks (No-Query). To address logical disconnects like boundary blurring, we propose Business Logic Alignment . In Query Generation , inspired by Hard Negative Mining ( Robinson et al. 2021 ) , we design “Adversarial Distractors” —options with subtle numerical or semantic alterations—to enforce precise discrimination. In Answer Generation , we employ SOP CoT to enforce: (1) Logic Explicitation , decomposing judgment into a verification pipeline (Entity \to Attribute \to Compliance); and (2) Correction-based Reasoning , mandating explicit clause citations for self-verification. Figure 3: Answer Correction Loop. Figure 4: Prompt Correction Loop. Complex Reasoning Tasks (With-Query). We target two domains. For Actuarial Math , we mitigate noisy free-form CoT (Fig. ) via: (1) Standardized Reasoning , aligning derivations with linear steps (similar to PoT) ( Chen et al. 2022 ) , yielding a +10.8% gain; and (2) an Answer Correction Loop (Self-Refine) ( Madaan et al. 2023 ) that iteratively executes a Generate-Verify-Reflect-Rewrite pipeline. For Underwriting & Claims , we handle implicit constraints via: (1) an APE-inspired Prompt Correction Loop ( Zhou et al. 2023c ) , refining prompts based on validation errors (Fig. ) to boost accuracy ( 80\!\to\!85 ); and (2) SOP-CoT Injection , enforcing a structured logic path (Info Extraction \to Risk ID \to Rule Detection \to Conclusion) to ensure long-tail robustness. 2.2.2 Hallucination Mitigation Data We adopt a phased approach, migrating the model to the insurance domain via low-hallucination QA and refusal samples. Ultimately, we optimize using GRPO with a Faithfulness reward model, mitigating reward hacking by incorporating real scenario queries. Atomic Capability Enhancement. We decompose RAG into four tasks: (1) Knowledge Boundary ID using CAIL2019 ( Duan et al. 2019 ) to train refusal when evidence is lacking; (2) Knowledge Selection via DRCD ( Shao et al. 2018 ) with distractors for precise location; (3) Summarization combining CNN/DailyMail ( See et al. 2017 ) and CASSum with RLAIF for factual distillation; and (4) Self-Check leveraging CHEF ( Hu et al. 2022 ) for post-generation hallucination correction. RAG Task Adaptation. We synthesize QA pairs from FAQs, using an LLM validator to route consistent samples to generation training and inconsistent ones to

Read the complete text