Policy-Governed LLM Routing with Intent Matching for Instrument Laboratories

Emmanuel A. Olowe, Danial Chitnis · 2026

Read Policy-Governed LLM Routing with Intent Matching for Instrument Laboratories by Emmanuel A. Olowe, Danial Chitnis (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-26955.txt.

Introduction

Olowe and Chitnis describe a routing and governance system for AI tutors in engineering labs, balancing help against learning. Routiium, an OpenAI-compatible gateway, manages multiple language-model backends with configurable prompts and logs, while EduRouter enforces per-lab budgets, approval workflows, and embedding-based question matching. In simulations from two labs, governed policies lifted challenge alignment and overlay adherence over ungoverned operation, and stretched the struggle window before scaffold hints appeared. A 100-query replay sent 75 percent of queries to a local model, cutting token costs by 66 percent.

Catalog record

CREATOREmmanuel A. Olowe, Danial Chitnis
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38880
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.26955

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Policy-Governed LLM Routing with Intent Matching for Instrument Laboratories Emmanuel A. Olowe, Danial Chitnis Abstract AI tutoring systems in engineering labs face a tension between providing sufficient assistance and preserving learning opportunities. Existing systems typically offer instructors limited control over assistance timing, content, or cost. This paper describes a routing and governance system for LLM-based lab assistance comprising two components: Routiium, an OpenAI-compatible gateway that manages multiple LLM backends with configurable prompt modifications and usage logging, and EduRouter, a policy-aware routing service that enforces per-lab budgets, approval workflows, and embedding-based question matching. We evaluated the system using trace-driven simulation calibrated from two engineering labs (LED characterization, RC circuit analysis) and a 100-query replay through live models. In simulations, governed policies (P1/P2) increased challenge-alignment index from 0.90 to 0.98 and overlay-adherence score from 0.69 to 0.87 compared to ungoverned operation (P0). The productive-struggle window metric increased from 1.4 to 3.6 simulated turns before high-scaffold hints appeared. In the 100-query replay, EduRouter routed 75% of queries to a local model, reducing token costs by 66% ($0.087 vs. $0.26 for all-premium routing) while maintaining canonical hit rate of 1.0 for the curated 89-intent question bank. We release Routiium, EduRouter, canonical-task tooling, and simulator configurations to support replication and future classroom studies. I Introduction Large language models (LLMs) are increasingly used as ad hoc tutors in engineering courses, including instrument-based laboratory work. In these settings, the same student question can be answered in ways that differ not only in correctness but also in pedagogical consequence: a minimal check that preserves student agency, a guided troubleshooting step, a worked-example fragment, or a complete solution. Classic tutoring and instructional science has long treated this as a control problem: how to balance information giving and withholding so that assistance supports learning without inducing dependence. Early work that introduced the notion of instructional scaffolding emphasized contingent support that is adjusted to learner need [ 34 ] . In intelligent tutoring systems (ITS), this tension is formalized as the assistance dilemma , where different help policies can change student engagement and learning outcomes, with no single policy dominating across contexts [ 12 , 30 ] . Current LLM tutoring deployments [ 14 , 26 ] typically provide instructors limited control over (i) when higher-scaffold help is allowed, (ii) what transformations are applied to prompts and responses, and (iii) how usage costs are bounded and audited. These limitations are operational as well as pedagogical: without explicit budgets, approval workflows, and transparency mechanisms, instructors cannot reliably enforce course policies (for example, restricting complete solutions during assessed steps) or explain system behavior when students challenge a decision. Recent reviews of LLMs in education describe both potential benefits and risks, including concerns about over-reliance and integrity, but do not prescribe implementable control mechanisms for lab-style tutoring at turn level [ 11 , 15 ] . In parallel, systems research on LLM routing has focused on reducing cost or latency by selecting among multiple models (for example via cascades or learned routers) [ 2 , 18 , 8 ] . However, these routers typically optimize for answer quality proxies and budget constraints, not for instructor-defined help policies such as “permit only conceptual guidance unless approved” or “cap complete-solution frequency per student.” Separately, retrieval and embedding methods provide a practical mechanism for mapping student questions to recurring intents (for example via sentence embeddings and dense retrieval) [ 24 , 5 , 10 ] . Yet existing educational deployments rarely connect embedding-based intent matching to auditable governance actions (approval queues, spend debits, policy overlays) in instrumented lab environments. This paper addresses the following gap: there is limited systems evidence on how to combine cost-aware multi-model routing with instructor-governed scaffolding policies and auditable prompt interventions for instrument laboratories . We describe a routing and governance system comprising two components. Routiium is an OpenAI-compatible gateway that fronts multiple LLM backends, applies configurable prompt modifications (overlays), and logs per-turn usage. EduRouter is a policy-aware routing service that enforces per-lab budgets, approval workflows, and embedding-based question matching against an instructor-curated library of common lab intents. Our design uses embedding similarity to match student queries to a canonical question library. When a match exceeds a configurable threshold, routing consults the matched entry’s metadata (preferred model tier, permitted hint level, maximum cost, overlay choice). When no match is available, routing falls back to heuristic selection. The goal is not to infer pedagogy from first principles, but to implement instructor-specified rules with traceable, testable behavior. We evaluate the system in two ways. First, we use trace-driven simulation calibrated from two electronics labs to measure whether routing and governance actions conform to specified policies under synthetic workloads. Second, we replay 100 curated queries through live model endpoints to measure cost, latency, and routing correctness relative to the canonical library. These evaluations characterize system behavior under the stated conditions; they do not measure learning outcomes, instructor workload reduction, or student satisfaction. Consistent with this scope, we treat our metrics as behavioral compliance measures rather than educational effectiveness measures [ 27 , 19 ] . Section X details threats to validity, including the constructed nature of the replay workload and the limited calibration data. This paper makes the following contributions: • Routiium, an OpenAI-compatible gateway enforcing configurable prompt overlays with per-turn logging and auditability. • EduRouter, a routing service implementing policy-aware model selection with embedding-based matching to an instructor-curated question library. • A set of governance-oriented metrics that characterize alignment between observed help behavior and instructor-specified policies, without asserting educational efficacy. • Synthetic and replay-based evaluations that quantify policy compliance, cost, and latency under the stated experimental constraints. • Open-source release of all components and evaluation tooling. II Background and Related Work II-A Scaffolding, the Assistance Dilemma, and Help-Seeking Work on instructional scaffolding emphasizes contingent support that is adjusted to learner need rather than uniformly increasing explanation detail [ 34 ] . In ITS research, this is operationalized as the assistance dilemma: policies that provide more help can improve short-term task completion while risking reduced learner engagement, and the conditions under which help is beneficial remain context dependent [ 12 ] . Comparative reviews of tutoring systems report substantial variance by domain and implementation, reinforcing that system designers should avoid assuming a single optimal help policy [ 30 ] . Our system does not propose a new theory of when help should be given; it implements instructor-specified help policies (hint levels, approvals, frequency caps) and provides instrumentation to audit compliance. II-B Teacher Agency and Classroom Orchestration Prior work on classroom orchestration treats teacher intervention as a first-class systems requirement, where tools should support monitoring and timely control actions rather than only automate tutoring [ 3 ] . Recent studies on teacher–AI complementarity similarly argue for interfaces and workflows that let teachers shape and audit automated support [ 7 ] . We operationalize this perspective for instrument labs by providing turn-level governance actions (approval queues, budgets, freezes) plus audit logs that expose routing rationale and applied prompt overlays. II-C Cost-Aware LLM Routing Several systems route queries across multiple LLM backends to reduce costs. FrugalGPT [ 2 ] cascades models from cheapest to most expensive, stopping when output confidence exceeds a threshold. RouteLLM [ 18 ] learns routing

Read the complete text