Portal UX Agent — A Plug-and-Play Engine for Rendering UIs from Natural-Language Specifications
Xinsong Li, Ning Jiang, Jay Selvaraj · 2025
Read Portal UX Agent — A Plug-and-Play Engine for Rendering UIs from Natural-Language Specifications by Xinsong Li, Ning Jiang, Jay Selvaraj (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-00843.txt.
Introduction
Xinsong Li, Ning Jiang, and Jay Selvaraj present the Portal UX Agent, a system that turns plain language requests into working user interfaces. Free form code generation is flexible but often breaks reliability, security, and design rules. Fully fixed interfaces are safe but cannot adapt.
Their middle path lets a language model plan the interface at a high level, while a deterministic renderer assembles it from approved components and layout templates under a strict schema. That keeps output auditable and reusable. The team also builds an evaluation framework mixing automatic checks on coverage, layout, and accessibility with a model judge for meaning and polish. In multi domain portal tests, the agent reliably produced coherent, usable interfaces.
Catalog record
| CREATOR | Xinsong Li, Ning Jiang, Jay Selvaraj |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 12 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-00843
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Portal UX Agent — A Plug-and-Play Engine for Rendering UIs from Natural-Language Specifications Xinsong Li, Ning Jiang, Jay Selvaraj Abstract The rapid appearance of large language models (LLMs) has led to systems that turn natural-language intent into real user interfaces (UIs). Free-form code generation maximizes expressiveness but often hurts reliability, security, and design-system compliance. In contrast, fully static UIs are easy to govern but lack adaptability. We present the Portal UX Agent, a practical middle way that makes bounded generation work: an LLM plans the UI at a high level, and a deterministic renderer assembles the final interface from a vetted set of components and layout templates. The agent maps intents to a typed composition—template and component specifications—constrained by a schema. This enables auditability, reuse, and safety while preserving flexibility. We also introduce a mixed-methods evaluation framework that combines automatic checks (coverage, property fidelity, layout, accessibility, performance) with an LLM-as-a-Judge rubric to assess semantic alignment and visual polish. Experiments on multi-domain portal scenarios show that the Portal UX Agent reliably turns intent into coherent, usable UIs and performs well on compositionality and clarity. This work advances agentic UI design by combining model-driven representations, plug-and-play rendering, and structured evaluation, paving the way for controllable and trustworthy UI generation. I Introduction LLM-powered systems can close the gap between what users say and what interfaces do, generating or adapting UIs from natural-language descriptions. But there is a persistent tension between expressivity and governance: unconstrained code generation is flexible but brittle; fully static UIs are reliable but not adaptable. We introduce the Portal UX Agent , an agentic system that translates natural-language descriptions into rendered portal UIs via a schema-bounded, slot-based composition model. The approach separates high-level planning (LLM) from low-level assembly (deterministic renderer), much like model-driven architecture (MDA) abstractions in software engineering. The LLM outputs a typed composition JSON that must validate against a component and template schema. A renderer then instantiates only vetted components, ensuring that all UIs are built from an auditable and reusable inventory. Contributions: • A novel agentic architecture for bounded UI generation that decouples semantic planning from deterministic assembly. • A plug-and-play engine and typed representation (template plus component specifications) that supports dynamic rendering without writing application code. • A mixed-methods evaluation framework integrating automatic checks with an LLM-as-a-Judge rubric for assessing intent alignment and UI quality. • An empirical study across multi-domain portal scenarios showing reliable intent translation and high scores on compositionality and clarity. Paper outline: Section II positions our work within generative UI and agentic UX literature. Section III formalizes the problem and the governance/expressivity trade-off. Section IV details the approach. Section V summarizes the prototype. Sections VI and VII present evaluation protocols and results. Section VIII discusses implications and limitations. Section X concludes. II Background and Related Work Generative and agentic user interfaces increasingly constrain LLMs to select and configure components rather than emit arbitrary code, improving predictability and validation [ 1 , 2 , 3 , 4 ] . Prior work demonstrates dynamic GUI synthesis within chats and workflows [ 1 ] and argues for structured, intermediate representations to capture design intent [ 2 ] . Surveys highlight both opportunity and risk in LLM-based UI agents, calling for evaluation frameworks that measure alignment, usability, accessibility, and robustness [ 3 , 5 , 6 ] . Our approach adopts the pragmatic design choice of bounded generation : intent is expressed in natural language but compiled into a schema-validated composition. This aligns with trends in industry and research to mitigate hallucination by operating over a fixed inventory of vetted components and templates [ 3 ] . The analogy to block-based assembly has been noted in creative systems where coarse-to-fine planning improves controllability [ 7 ] . We complement this with a principled evaluation pipeline that combines automatic checks with rubric-based LLM judgment [ 8 ] . III Problem Statement and System Model We formalize intent-to-UI generation as a mapping from natural-language specifications to a bounded UI composition. Let s\in\mathcal{S} denote a natural-language specification (intent). The agent computes a typed composition c\in\mathcal{C} and a rendered UI y\in\mathcal{Y} via c=f_{\theta}(s),\quad y=g(c), (1) where f_{\theta} is an LLM-based planner constrained to produce c that validates against a schema \Sigma (templates, slots, component types, and props), and g is a deterministic renderer that instantiates only vetted components. Bounded generation enforces governance through: • Schema compliance: c\models\Sigma must hold; invalid compositions are rejected or repaired. • Component inventory: types and props are drawn from a vetted library; no arbitrary code is emitted. • Slot-based layout: placements occur in declared template slots, preserving hierarchy and design-system rules. Evaluation targets both structural correctness and experiential quality. Automatic metrics operate on the intent specification and the rendered UI tree; rubric-based judgments capture semantics and polish. IV Approach and Methodology IV-A Typed Composition and Templates The LLM proposes a composition consisting of a template identifier and a set of component specifications, each with a type, slot, and typed props. The schema \Sigma guarantees that compositions are renderable and auditable. This model-driven representation functions as a platform-independent UI plan; rendering realizes a platform-specific view. IV-B Planner & Deterministic Renderer The planner uses constrained prompting to emit JSON compatible with \Sigma . The renderer then loads the template, fills slots with validated components, and produces the final UI deterministically. This decoupling preserves creativity at the planning layer while ensuring predictable assembly and safety at render time [ 2 ] . IV-C Evaluation: Automatic Checks and LLM-as-a-Judge We combine objective checks with rubric-based model judgments. Given an intent specification (Expected) and a rendered UI tree and snapshot (Actual), we compute: \displaystyle\mathrm{AutoScore} \displaystyle=0.35\,S_{\text{cov}}+0.20\,S_{\text{prop}}+0.10\,S_{\text{data}} (2) \displaystyle\quad+0.15\,S_{\text{layout}}+0.10\,S_{\text{a11y}}+0.10\,S_{\text{perf}}, (3) where S_{\text{cov}} measures intent coverage, S_{\text{prop}} property fidelity, S_{\text{data}} grounding, S_{\text{layout}} layout and hierarchy, S_{\text{a11y}} accessibility, and S_{\text{perf}} performance. An LLM judge provides rubric scores for intent alignment, semantic correctness, accessibility signals, visual polish, and an overall verdict [ 8 ] . When ambiguity remains, a lightweight human checklist adjudicates edge cases, and suggested diffs guide regeneration. V Prototype Summary We implemented a plug-and-play agent that exposes a simple interface to request UIs from natural-language prompts and structured inputs. The system compiles intents into a typed composition and renders UIs from a reusable inventory of vetted components and templates. For interoperability with tool ecosystems, the agent can be wrapped behind a standardized model-context protocol endpoint, but infrastructure details are intentionally kept orthogonal to the research questions. The primary focus is on the representation, bounded generation principle, and evaluation methodology rather than on any particular framework or runtime. VI Experiments and Evaluation VI-A Datasets and Scenarios We evaluate on a set of multi-domain portal scenarios (e.g., analytics dashboards, boards, and content portals). Each scenario provides a textual description of required regions and components (KPIs, filters, tables, charts, boards). VI-B Protocols and Baselines We run a five-step pipeline: (1) parse scenario intent; (2) generate typed composition; (3) render the UI; (4) compute automatic metrics; (5) obtain rubric-based LLM judgments. For context, we compare against an unconstrained prompting baseline that emits free-form code; this baseline tends to increase hallucination and reduce governance. VI-C Metrics We report per-dimension means for structural and experiential quality (higher is better). Automatic metrics include S_{\text{cov}} , S_{\text{prop}} , S_{\text{layout}} ,