Task Cascades for Efficient Unstructured Data Processing Journal: PACMMOD Volume: 4 1 (SIGMOD) 88 2 DOI: 10.1145/3786702 CCS: Information systems Data management systems CCS: Applied computing Document management and text processing CCS: Computing methodologies Natural language processing

Shreya Shankar, Sepanta Zeighami, Aditya G. Parameswaran · 2026

Read Task Cascades for Efficient Unstructured Data Processing Journal: PACMMOD Volume: 4 1 (SIGMOD) 88 2 DOI: 10.1145/3786702 CCS: Information systems Data management systems CCS: Applied computing Document management and text processing CCS: Computing methodologies Natural language processing by Shreya Shankar, Sepanta Zeighami, Aditya G. Parameswaran (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-05536.txt.

Introduction

Shreya Shankar and 2 co-authors introduce task cascades, a 2026 framework for cutting the cost of running language models over large document collections. Databases increasingly let users ask natural-language questions over text columns, row by row. A cheap model can handle easy cases, with hard ones passed to an expensive oracle.

Standard cascades vary only the model. Task cascades also vary which part of a document is read and which simpler, related operation is run first. An agent generates hundreds of candidate tasks and assembles them into a cascade. The authors prove optimal selection is intractable, give an iterative builder, and add an accuracy-guarantee extension. Across eight tasks, costs fell by about a third on average.

Catalog record

CREATORShreya Shankar, Sepanta Zeighami, Aditya G. Parameswaran
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE118 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-05536

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Task Cascades for Efficient Unstructured Data Processing Journal: PACMMOD Volume: 4 1 (SIGMOD) 88 2 DOI: 10.1145/3786702 CCS: Information systems Data management systems CCS: Applied computing Document management and text processing CCS: Computing methodologies Natural language processing Shreya Shankar, Sepanta Zeighami, Aditya G. Parameswaran Abstract . Modern database systems allow users to query or process unstructured text or document columns using LLM-powered functions. Users can express an operation in natural language (e.g., “identify if this review mentions billing issues” ), with the system executing the operation on each document, in a row-by-row fashion. One way to reduce cost on a batch of documents is to employ the model cascade framework: a cheap proxy model processes each document, and only uncertain cases are escalated to a more accurate, expensive oracle. However, model cascades miss important optimization opportunities; for example, often only part of a document is needed to answer a query, or other related, but simpler operations (e.g., “is the review sentiment negative?”, “does the review mention money?” ) can be handled by cheap models more effectively than the original operation, while still being correlated with it. We introduce the task cascades framework, which generalizes model cascades by varying not just the model, but also the document portion and operation at each stage. Our framework uses an LLM agent to generate simplified, decomposed, or otherwise related operations and selects the most relevant document portions, constructing hundreds of candidate tasks from which it assembles a task cascade. We show that optimal cascade selection is intractable via reduction from Minimum Sum Set Cover , but our iterative approach constructs effective cascades. We also provide an extension that offers statistical accuracy guarantees: the resulting cascade meets a user-defined accuracy target (with respect to the oracle) up to a bounded failure probability. Across eight real-world document processing tasks at a 90% target accuracy, task cascades reduce end-to-end cost by an average of 36% compared to model cascades at production scale. † † cc-license: by 1. Introduction Figure 1 . Task vs. model cascade for determining whether a Supreme Court opinion overturns a lower court decision in Example 1.1 . In a task cascade (left), each stage (or task) is defined by a document fraction ( f ), a model ( m ), and an operation ( o ). Surrogate operations may be reused at different document fractions (as in tasks 1 and 3). The final oracle task applies the original user-specified operation ( o^{*} ) on the full document ( f=1.0 ) with the oracle model ( m= gpt-4o) if prior tasks cannot confidently resolve the input. In contrast, a model cascade (right) applies the same operation o^{*} to the full document at each stage, varying only the model. Large language models (LLMs) are increasingly being integrated as operators in data management systems, enabling users to analyze unstructured text columns more easily. This line of work spans SQL extensions to databases, providing LLM-powered map and filter operations ( 67 ; 16 ; 21 ; 55 ) , as well as Python-based declarative frameworks for executing pipelines of LLM-powered operators ( 41 ; 49 ; 53 ; 65 ; 1 ) . In all cases, users describe an operation in natural language, with the system executing it by invoking an LLM on each text value—hereafter referred to as a document —in a row-by-row fashion. Consider the following example: Example 1.1 (Supreme Court Opinion Analysis). Suppose a legal analyst is working with a large collection of Supreme Court opinions. The analyst’s goal is to determine, for each opinion, whether it overturns a lower court decision. In this setting, each document is a long, complex legal text, and the analyst poses an operation in natural language: “Does this opinion overturn a lower court decision?” State-of-the-art LLMs like GPT-4o and GPT-4.1 can achieve high accuracy on tasks such as Example 1.1 . However, commercial LLM APIs charge by the number of input and output tokens, making large-scale analysis costly. As a result, users look for ways to cut inference costs without losing much of the accuracy provided by the best models. They are often willing to accept a modest drop in accuracy—e.g., targeting 90% of GPT-4o’s accuracy—in exchange for significant savings ( 52 ; 49 ) . Although cheaper models exist (such as GPT-4o-mini, which is nearly 17\times cheaper than GPT-4o), they often deliver much lower accuracy. Prior Work and Limitations. The standard approach to reduce costs of map and filter operations with machine learning models, while preserving output accuracy, is to use a model cascade ( 34 ; 2 ; 36 ; 51 ; 37 ) ; with recent systems all adopting it as-is for LLM-powered operators ( 9 ; 49 ; 70 ) . Here, the system first applies a cheaper proxy model to each input document. If the proxy’s confidence in its prediction exceeds a system-selected confidence threshold, the system accepts the proxy’s output. Otherwise, the system escalates the input to a more accurate but more expensive oracle model (e.g., the best available LLM). Users specify a performance target (e.g., 90% of the oracle’s accuracy), and the system tunes the confidence thresholds to meet this target while minimizing cost. However, the model cascades framework, by focusing only on swapping models, underutilizes optimization opportunities. For instance, when no cheaper model can perform the operation accurately, documents will end up being processed by the oracle. In fact, on medical or legal tasks, cheaper models may have as little as 30% of the accuracy of expensive models ( 63 ; 17 ; 47 ) , with the gap being even wider for long-context reasoning ( 68 ; 42 ) . Moreover, model cascades always process the full document, even though many operations require only a small, relevant portion of the text. Change the Task, Not Just the Model. Our key insight is that effective cascades should vary the operation as well as the portion of the document , in addition to the model. Earlier stages of the cascade can execute any operation that is easier for cheaper models—such as a prerequisite check, a decomposed sub-task, or any related operation that is correlated with the original. We refer to such operations as surrogate operations: variants of the original operation that are easier for a proxy model to perform. In Example 1.1 , rather than having a proxy model decide if an opinion overturns a lower court decision, we can have it check whether the opinion mentions a lower court at all. If not, we don’t need to invoke the oracle. Another complementary strategy to improve proxy model accuracy is to prune irrelevant context from each document before inference. For example, instead of providing the entire document to the LLM, we can supply only the sections that are semantically relevant to the operation—such as paragraphs containing “overturn,” “judgment vacated,” or their synonyms. By eliminating unrelated text, the proxy model is more likely to perform the operation both correctly and with lower cost ( 69 ; 58 ) . In this paper, we introduce the notion of a task cascade : an ordered sequence of tasks , each task defined by an operation, an LLM, and a selected fraction of the document

Read the complete text