LLM-Rosetta : A Hub-and-Spoke Intermediate Representation for Cross-Provider LLM API Translation
Peng Ding · 2026
Read LLM-Rosetta : A Hub-and-Spoke Intermediate Representation for Cross-Provider LLM API Translation by Peng Ding (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-09360.txt.
Introduction
Ding presents LLM-Rosetta, a middle model for cross-vendor language model API switching. Vendors each expose private API formats, splitting the world and needing squared two-way bridges to switch vendors. Major APIs share a common meaning core, so the challenge is syntax change, not deep meaning mismatch. LLM-Rosetta captures that core in a 9-type content model and 10-type stream event schema, giving two-way switching with lossless round-trip match and live use at Argonne.
Catalog record
| CREATOR | Peng Ding |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 44839 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.09360
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
LLM-Rosetta : A Hub-and-Spoke Intermediate Representation for Cross-Provider LLM API Translation Peng Ding Abstract The rapid proliferation of Large Language Model (LLM) providers—each exposing proprietary API formats—has created a fragmented ecosystem where applications become tightly coupled to individual vendors. Switching or bridging providers requires O ( N 2 ) O(N^{2}) bilateral adapters, impeding portability and multi-provider architectures. We observe that despite substantial syntactic divergence, the major LLM APIs share a common semantic core: the practical challenge is the combinatorial surface of syntactic variations, not deep semantic incompatibility. Based on this finding, we present LLM-Rosetta , an open-source translation framework built on a hub-and-spoke Intermediate Representation ( IR ) that captures the shared semantic core—messages, content parts, tool calls, reasoning traces, and generation controls—in a 9-type content model and 10-type stream event schema. A modular Ops-composition converter architecture enables each API standard to be added independently. LLM-Rosetta supports bidirectional conversion (provider ↔ \leftrightarrow IR ↔ \leftrightarrow provider) for both request and response payloads, including chunk-level streaming with stateful context management. We implement converters for four API standards (OpenAI Chat Completions, OpenAI Responses, Anthropic Messages, and Google GenAI), covering the vast majority of commercial providers. Empirical evaluation demonstrates lossless round-trip fidelity, correct streaming behavior, and sub-100 µs conversion overhead—competitive with LiteLLM’s single-pass approach while providing bidirectionality and provider neutrality. LLM-Rosetta passes the Open Responses compliance suite and is deployed in production at Argonne National Laboratory. Code is available at https://github.com/Oaklight/llm-rosetta . Keywords LLM \cdot API Translation \cdot Intermediate Representation \cdot Interoperability \cdot Streaming \cdot Multi-Provider 1 Introduction Large Language Models (LLMs) ( Achiam et al. 2023 ; Gemini Team et al. 2023 ) are increasingly accessed through cloud-hosted APIs, yet the ecosystem lacks a universal wire format. OpenAI’s Chat Completions ( OpenAI 2024 ) , Anthropic’s Messages ( Anthropic 2024a ) , Google’s Generative AI ( Google 2024 ) , and the newer OpenAI Responses API ( OpenAI 2025 ) each define their own schemas for messages, tool calls, streaming, and generation controls. This divergence forces application developers to write provider-specific glue code, and organizations evaluating multiple models face a combinatorial integration burden. The O(N^{2}) problem. Given N providers, naïve pairwise translation requires \binom{N}{2} bilateral adapters. Each adapter must handle request construction, response parsing, streaming event mapping, and tool-call serialization—all of which drift as providers evolve their APIs. In practice, most projects either lock into a single vendor or adopt heavyweight SDK wrappers that hide—but do not solve—the underlying format mismatch. Hub-and-spoke as O(N) solution. The hub-and-spoke pattern—routing all conversions through a single Intermediate Representation ( IR )—is well established in compiler infrastructure ( Lattner and Adve 2004 ) and data interchange ( Apache Software Foundation 2016 ) , where it reduces M\times N adapters to M+N . LLM API translation is a structurally simpler domain (dict-to-dict mapping rather than semantic-preserving program transformation), but the same combinatorial argument applies: with N providers and multiple feature dimensions, pairwise adapters grow quadratically. We apply the hub-and-spoke principle to this domain. Contributions. We present LLM-Rosetta , an open-source framework that introduces: 1. An empirical characterization of API divergence : despite substantial syntactic differences, the four major LLM providers share a common semantic core that can be captured by a 9-type content model and 10-type stream event schema. The practical difficulty lies not in deep semantic gaps but in the combinatorial surface of syntactic variations across providers and feature dimensions ( section 3.2 ). 2. A typed Intermediate Representation ( IR ) and Ops-composition converter architecture that exploit this observation: since divergence is predominantly syntactic, a provider-neutral IR can faithfully represent all four formats, and each provider adapter can be assembled from four orthogonal operations modules (content, message, tool, config) with effort independent of existing providers ( section 3 ). 3. Bidirectional conversion with streaming support : request and response payloads are translated in both directions (provider \leftrightarrow IR \leftrightarrow provider), and chunk-level streaming is handled through ten typed event kinds with stateful context management ( section 4 ). 4. An empirical evaluation demonstrating lossless round-trip fidelity, correct streaming behavior, and sub-millisecond conversion overhead, validated by 1,364 tests and the Open Responses compliance suite, and deployed in production at Argonne National Laboratory ( section 5 ). LLM-Rosetta currently supports four API standards—OpenAI Chat Completions, OpenAI Responses, Anthropic Messages, and Google Generative AI—which collectively cover the vast majority of commercial LLM providers, as most adopt one of these wire formats. The code is released under the MIT license at https://github.com/Oaklight/llm-rosetta . Paper organization. Section 2 surveys related work. Section 3 presents the IR design and converter architecture. Section 4 describes the implementation, including streaming and the gateway proxy. Section 5 evaluates round-trip fidelity, streaming correctness, and performance overhead. Section 6 discusses limitations and future directions. Section 7 concludes. 2 Related Work 2.1 LLM API Ecosystem The landscape of LLM APIs has evolved rapidly since the release of GPT-3 ( Brown et al. 2020 ) . OpenAI’s Chat Completions API ( OpenAI 2024 ) established an early de facto standard based on role-tagged messages ( system , user , assistant ), but subsequent providers diverged. Anthropic’s Messages API ( Anthropic 2024a ) introduced a separate system parameter and block-typed content arrays. Google’s Generative AI API ( Google 2024 ) adopted a contents / parts schema with user / model roles. OpenAI itself introduced the Responses API ( OpenAI 2025 ) , replacing the chat message paradigm with an items-based model where tool calls, reasoning, and text output are sibling items rather than nested content parts. This fragmentation extends to tool calling ( Schick et al. 2023 ; Patil et al. 2023 ; Qin et al. 2023 ) (function definitions and invocation formats), streaming (SSE event schemas), multi-modal content ( Liu et al. 2024 ) (image, audio, file encoding), and generation controls (temperature, top-p, reasoning budgets ( Wei et al. 2022 ) ). Table 1 summarizes key differences across providers. Table 1: API format divergence across major LLM providers. Aspect OpenAI Chat Anthropic Google GenAI OpenAI Responses Message unit messages[] messages[] contents[] input[] items System prompt role in array top-level param role in array instructions Content model string or parts block array parts[] items with type Tool calls tool_calls[] content block functionCall item with type Streaming delta chunks event types candidate deltas response events Reasoning — thinking thought reasoning item 2.2 SDK Wrappers and Abstraction Layers Several projects attempt to unify LLM access at different levels of abstraction. Multi-provider frameworks. LangChain ( LangChain, Inc. 2024 ) and Microsoft’s Semantic Kernel ( Microsoft 2024 ) are the two most widely adopted multi-provider LLM frameworks. Both provide high-level abstractions (chains, agents, planners) that internally dispatch to provider-specific SDK clients. Their multi-provider support operates at the application level: each provider integration is a separate adapter class that maps framework-level abstractions to native API calls. This design prioritizes developer ergonomics for building LLM applications but does not expose a reusable, format-level translation layer—cross-provider conversion of raw API payloads is not a supported use case. SDK-level proxies. LiteLLM ( BerriAI 2024 ) provides an OpenAI-compatible proxy that translates requests at the SDK level, mapping all providers into the OpenAI Chat Completions format. While widely adopted, this approach uses a single provider’s schema as