ComplianceNLP: Knowledge-Graph-Augmented RAG for Multi-Framework Regulatory Gap Detection
Dongxin Guo, Jikun Wu, Siu Ming Yiu · 2026
Read ComplianceNLP: Knowledge-Graph-Augmented RAG for Multi-Framework Regulatory Gap Detection by Dongxin Guo, Jikun Wu, Siu Ming Yiu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-23585.txt.
Introduction
Guo, Wu, and Yiu build ComplianceNLP, a system that watches regulatory change for banks and flags gaps against their internal policies. A knowledge-graph-augmented retrieval pipeline grounds answers in nearly 13,000 provisions across three major rulebooks, while obligation extraction and severity-aware gap scoring complete the chain. It beats a strong GPT-4o retrieval baseline on gap detection and held up well in a four-month live deployment.
Catalog record
| CREATOR | Dongxin Guo, Jikun Wu, Siu Ming Yiu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 70816 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.23585
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ComplianceNLP: Knowledge-Graph-Augmented RAG for Multi-Framework Regulatory Gap Detection Dongxin Guo, Jikun Wu, Siu Ming Yiu Abstract Financial institutions must track over 60,000 regulatory events annually across fragmented jurisdictions, a volume that overwhelms manual compliance teams. The industry has paid over $300 billion in fines and settlements since the 2008 financial crisis. We present ComplianceNLP , an end-to-end system that automatically monitors regulatory changes, extracts structured obligations, and identifies compliance gaps against institutional policies. The system integrates three components: (1) a knowledge-graph-augmented RAG pipeline grounding generations in a regulatory knowledge graph of 12,847 provisions across SEC, MiFID II, and Basel III; (2) multi-task obligation extraction combining NER, deontic classification, and cross-reference resolution over a shared LEGAL-BERT encoder; and (3) compliance gap analysis that maps obligations to internal policies with severity-aware scoring. On our benchmark, ComplianceNLP achieves 87.7 F1 on gap detection, outperforming GPT-4o+RAG by +3.5 F1, with 94.2% grounding accuracy ( r = 0.83 r{=}0.83 vs. human judgments) and 83.4 F1 under realistic end-to-end error propagation. Ablations show that knowledge-graph re-ranking contributes the largest marginal gain (+4.6 F1), confirming that structural regulatory knowledge is critical for cross-reference-heavy tasks. Domain-specific knowledge distillation (70B → \to 8B) combined with Medusa speculative decoding yields 2.8 × 2.8\times inference speedup, enabled by an empirical finding that regulatory text’s low entropy ( H = 2.31 H{=}2.31 bits vs. 3.87 3.87 general text) produces 91.3% draft-token acceptance rates. In four months of parallel-run deployment processing 9,847 updates at a financial institution, the system achieved 96.0% estimated recall and 90.7% precision, with a 3.1 × 3.1\times sustained analyst efficiency gain. We report deployment lessons spanning trust calibration, GRC integration, and distributional shift monitoring that inform future regulated-domain NLP deployments. 1 Introduction Compliance with financial regulation has become a substantial operational task for banks. A single multinational bank may be subject to hundreds of regulatory frameworks across dozens of jurisdictions ( 12 ) . Over 60,000 regulatory events were tracked across 190 countries in 2022 ( 38 ) , far exceeding the capacity of human compliance teams. Global banks have paid over $300 billion in regulatory fines and settlements since the 2008 financial crisis ( 5 ) . Despite clear demand, automated regulatory compliance remains underserved within NLP. Legal NLP has advanced benchmarks and domain-adapted models ( 16 ; 10 ; 8 ) , and the COLING 2025 Regulations Challenge ( 14 ) highlighted significant gaps in LLM performance on financial regulatory tasks. Yet no end-to-end system bridges these advances to deliver production-ready compliance monitoring; commercial GRC platforms such as Ascent RegTech and Wolters Kluwer OneSumX still rely heavily on rule-based approaches with manual curation ( 21 ) (Appendix A ). We introduce ComplianceNLP , a system that automatically monitors regulatory changes, extracts structured obligations, and identifies compliance gaps against internal policies. The system currently covers three major frameworks (SEC, MiFID II, Basel III), representing approximately half of the annual regulatory update volume. Our approach addresses three challenges: (1) regulatory grounding : LLMs are prone to hallucination ( 39 ) , which we mitigate through KG-augmented RAG achieving 94.2% grounding accuracy ( r{=}0.83 vs. human; § 6.1 ); (2) multi-framework obligation extraction : prior systems target single frameworks ( 7 ; 4 ) , while we jointly handle three; and (3) production-grade latency : we achieve sub-second p50 inference via knowledge distillation ( 15 ) and Medusa speculative decoding ( 6 ) . The system achieves 87.7 gap detection F1 at the evaluation threshold ( \delta{=}0.6 ) and 85.3 macro F1 at the recall-optimized deployment threshold ( \delta{=}0.45 ), where Full Gap recall reaches 91.2%. Our contributions are: 1. An end-to-end regulatory compliance monitoring system integrating KG-augmented RAG, multi-task obligation extraction, and automated gap analysis across three frameworks, with 83.4 F1 under realistic error propagation and two independent additive analyses isolating each module’s contribution. 2. A Regulatory Knowledge Graph linking 12,847 provisions with 94.7% edge precision and 87.3% estimated recall. 3. A deployment analysis including production metrics from 4 months of parallel-run operation (9,847 updates, estimated 96.0% recall), cost modeling, and five lessons learned. 4. A production optimization pipeline where domain-specific distillation (70B \to 8B) and Medusa speculative decoding yield 2.8\times combined speedup, with the finding that regulatory text’s low entropy enables 91.3% Medusa acceptance rates. 2 Related Work Legal and regulatory NLP. Legal NLP benchmarks including LegalBench ( 16 ) , LexGLUE ( 10 ) , and CUAD ( 18 ) , together with domain models such as LEGAL-BERT ( 8 ) and the MultiLegalPile ( 29 ) , provide foundations for legal language understanding ( 2 ; 24 ) . For regulatory compliance specifically, Doc2Doc retrieval ( 9 ) identified limitations of text similarity for legislation, ObliQA/RIRAG ( 4 ) introduced regulatory QA, and DERECHA ( 7 ) demonstrated GDPR compliance checking at 89.1% precision but assumes pre-structured policy clauses and targets a single framework, whereas ComplianceNLP jointly handles three frameworks end-to-end from raw regulatory text and reports comparable precision (90.7%) at production scale (Table 4 ). Most closely related, 34 propose a RAG-based compliance checker built on an eventic graph; we differ in (i) spanning three regulatory frameworks rather than a single corpus, (ii) grounding retrieval in a typed regulatory knowledge graph rather than purely embedding-based vector store, (iii) learning end-to-end obligation extraction rather than assuming structured regulatory inputs, and (iv) reporting four months of parallel-run deployment evidence. 32 addressed deontic modality classification. None handles multi-framework obligation extraction with production deployment. Financial LMs, RAG, and efficient inference. Financial domain models ( 1 ; 40 ; 42 ; 41 ; 28 ) provide domain adaptation foundations. RAG ( 26 ) grounds LLM outputs, with KG-enhanced variants improving factual QA ( 3 ) , multi-hop reasoning ( 33 ) , and corpus-level queries ( 13 ) . LegalBench-RAG ( 30 ) and CLERC ( 19 ) benchmark legal retrieval, and LexKeyPlan ( 36 ) augments legal-text generation with anticipatory keyphrase planning to align retrieval with intended future content. For hallucination mitigation, MiniCheck ( 37 ) offers efficient fact-checking, the FACTS benchmark ( 20 ) evaluates long-document factuality, CoCoLex ( 35 ) enforces faithfulness via confidence-guided copy-based decoding for grounded legal generation, and 22 report industry grounding lessons. For efficient inference, Medusa ( 6 ) extends speculative decoding ( 25 ; 11 ) with multiple heads, and MiniLLM ( 15 ) enables effective distillation. Most relevant, 43 demonstrated Medusa+KD at Booking.com, a setup we adapt for compliance with domain-specific findings on acceptance rates. 3 The ComplianceNLP System Figure 1 presents the pipeline: (i) ingestion and indexing of regulatory documents into the RKG and vector store; (ii) multi-task obligation extraction; and (iii) compliance gap analysis mapping extracted obligations against internal policies. The system uses two distinct model families sequentially: a LEGAL-BERT-based extraction module (§ 3.2 ) produces structured obligations, which feed the LLaMA-3-based generator (§ 3.3 ) for gap analysis. The two models share no parameters. Full pseudocode and notation are in Appendix B . Regulatory Documents Chunking + Embedding Vector Store Regulatory KG Regulatory NER Deontic Classifier Cross-Ref Resolver Structured Obligations Internal Policies Obligation–Policy Alignment Gap Severity Scoring Compliance Gap Report Figure 1: Overview of the ComplianceNLP pipeline: regulatory documents are ingested into a vector store and knowledge graph