Specifications: The missing link to making the development of LLM systems an engineering discipline

Ion Stoica, Matei Zaharia, Joseph Gonzalez, Ken Goldberg, Koushik Sen, Hao Zhang, Anastasios N. Angelopoulos, Shishir G. Patil, Lingjiao Chen, Wei-Lin Chiang, Jared Q. Davis · 2024

Read Specifications: The missing link to making the development of LLM systems an engineering discipline by Ion Stoica, Matei Zaharia, Joseph Gonzalez, Ken Goldberg, Koushik Sen, Hao Zhang, Anastasios N. Angelopoulos, Shishir G. Patil, Lingjiao Chen, Wei-Lin Chiang, Jared Q. Davis (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-05299.txt.

Introduction

Ion Stoica and ten co-authors argue that language model systems will stay craft projects until their parts come with specifications. Their 2024 position paper notes that past revolutions, from cars to software, scaled by joining parts whose behavior was pinned down in advance. Each part could then be tested, swapped, and debugged alone.

Language components resist that treatment: their inputs and outputs are free text, broad and ambiguous by design. The authors survey the ground gained so far, in structured outputs, process supervision, and test-time computing. They then sketch research paths toward specs for agents and other model parts. Clear contracts, they argue, are the missing link between demos and dependable systems. The paper is a call to treat that gap as a first-class research problem.

Catalog record

CREATORIon Stoica, Matei Zaharia, Joseph Gonzalez, Ken Goldberg, Koushik Sen, Hao Zhang, Anastasios N. Angelopoulos, Shishir G. Patil, Lingjiao Chen, Wei-Lin Chiang, Jared Q. Davis
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE86 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-05299

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Specifications: The missing link to making the development of LLM systems an engineering discipline Ion Stoica, Matei Zaharia, Joseph Gonzalez, Ken Goldberg, Koushik Sen, Hao Zhang, Anastasios N. Angelopoulos, Shishir G. Patil, Lingjiao Chen, Wei-Lin Chiang, Jared Q. Davis Abstract Despite the significant strides made by generative AI in just a few short years, its future progress is constrained by the challenge of building modular and robust systems. This capability has been a cornerstone of past technological revolutions, which relied on combining components to create increasingly sophisticated and reliable systems. Cars, airplanes, computers, and software consist of components—such as engines, wheels, CPUs, and libraries—that can be assembled, debugged, and replaced. A key tool for building such reliable and modular systems is specification : the precise description of the expected behavior, inputs, and outputs of each component. However, the generality of LLMs and the inherent ambiguity of natural language make defining specifications for LLM-based components (e.g., agents) both a challenging and urgent problem. In this paper, we discuss the progress the field has made so far—through advances like structured outputs, process supervision, and test-time compute—and outline several future directions for research to enable the development of modular and reliable LLM-based systems through improved specifications. 1 Introduction Software has been one of the main drivers of economic growth over the past several decades, a trend famously captured by Andreessen in his influential blog post, “Why software is eating the world,” more than a decade ago [ 1 ] . More recently, AI—particularly Large Language Models (LLMs)—has emerged as the next revolution, poised to disrupt the very software that has been "eating the world" [ 2 , 3 , 4 ] . However, to fully realize this vision, we need to build LLM-based systems with the same level of reliability and rigor as those found in established engineering disciplines, such as control theory, mechanical engineering, and software engineering. One key tool that has historically enabled the rapid growth of engineering disciplines is specification . Specifications describe the expected behavior, inputs, and outputs of a system. These specifications have different levels of precision and come in many forms, including formal specifications, product requirements documents (PRDs), and user manuals. Specifications help developers in several ways: (1) decomposing complex systems into smaller components, (2) reusing existing components when possible, (3) verifying that a system works as intended (i.e., it meets its specification), (4) fixing the system when it does not, and (5) creating systems capable of making decisions without human intervention. Figure 1 illustrates how specifications enable these five properties in the automotive industry. Figure 1: A car consists of many parts (modularity), where some parts are shared with other cars (reusability). When something goes wrong, the mechanic diagnoses the problem (verifiability) and then fixes it (debuggability). Furthermore, some functions of the car don’t require human intervention (automated decision making). To enable these properties we need detailed schematics and detailed descriptions of how each of these components works (specifications). In this paper, we distinguish between two types of specifications: (i) a description of what the task should do, which we call a statement specification , and (ii) a description that enables a user to verify task solutions (outputs), which we call a solution specification . Essentially, statement specifications answer the question, "What should the task accomplish?" while solution specifications answer the question, "How can one verify that the task’s solution adheres to the statement specification?" In traditional software systems, the statement specification can be captured in a Product Requirements Document (PRD), whereas the solution specification can be expressed as input-output tests (e.g., unit tests). In formal frameworks like Coq/Gallina [ 5 , 6 ] , the statement specification is represented by a formal specification, and the solution specification is captured by a proof demonstrating that the generated code is correct. When building a radio, the statement specification is represented by the circuit schematics, while the solution specification is defined by the expected voltage and amperage at different points in the circuit. Finally, in some cases, the statement and solution specification coincide, e.g., the task, "solve x^{3}-x=0 ", specifies both what the task should do (i.e., solve the equation) and how a solution should be verified—by substituting the solution back into the equation to check the equality. Defining good specifications is challenging. Apart from formal specifications, most specifications contain some degree of ambiguity. For example, it is notoriously difficult to describe and test for every possible behavior of a software system [ 7 ] . That said, LLMs are even more ambiguous than software systems, and this has significant implications. With LLMs, users provide specifications through natural language prompts. While natural language is highly general and flexible, it often encourages users to specify ambiguous, ill-defined tasks. 1 1 1 This should come as no surprise, as natural language was developed by humans to communicate with one another. Humans excel at handling ambiguity by relying on rich context to complement language, including facial expressions, tone of voice, and shared historical and cultural backgrounds. Additionally, humans often ask clarifying questions to resolve ambiguity. While encouraging ambiguity has been one of the main strengths of LLMs, this has also led to spectacular failures, such as the well-known "hallucinations", where LLMs generate incorrect answers to user input prompts [ 8 , 9 , 10 ] . As illustrated by "hallucinations," the lack of clear specifications makes it challenging to build reliable systems . If a task is not well-specified, how can one even determine whether the task is being performed correctly? Moreover, this ambiguity complicates the development of modular systems composed of multiple LLM components (e.g., agents). Without a clear understanding of what each component should do, how can one define the behavior of a system that integrates these components? As we will argue in Section 2 , the challenge of building reliable, modular systems is significantly hindering the growth of the LLM ecosystem. Finally, in the absence of well-defined solution specifications, LLM-based applications (e.g., Q&A, customer support, copilot tools) often depend on humans to evaluate the quality of their outputs, as essential context is frequently missing from the task specifications. This dependence makes such applications unsuitable for fully automated decision-making without human intervention. Failures Description & Impact Possible fixes Wrong policy An Air Canada customer service chatbot provided wrong policy information about a ticket being eligible for refund. Eventually, Air Canada was ordered to honor the erroneous refund policy [ 11 ] . Specify policies to verify the answers. Ask customers for clarifications to disambiguate their questions, if needed. Wrong offer A Chevrolet customer service chatbot agreed to sell to a customer a late model for $1 [ 12 ] . Specify the rules for a valid offer (e.g., price \geq threshold), and monitor generated offers. Wrong health advice LLMs provided incorrect and harmful health advice [ 10 ] . Specify the rules for “safe” health advice, and use it to validate the advice. Bad code An LLM generated code that appears plausible but is functionally incorrect or references non-existent libraries and functions [ 9 ] . Provide or generate comprehensive tests to validate the generated code. Data leakage LLMs reproduced proprietary code or personal data present in their training

Read the complete text