EchoChain: A Full-Duplex Benchmark for State-Update Reasoning Under Interruptions

Smit Nautambhai Modi, Gandharv Mahajan, Marc Wetter, Randall Welles · 2026

Read EchoChain: A Full-Duplex Benchmark for State-Update Reasoning Under Interruptions by Smit Nautambhai Modi, Gandharv Mahajan, Marc Wetter, Randall Welles (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-16456.txt.

Introduction

Modi, Mahajan, Wetter and Welles built EchoChain, a benchmark for voice assistants that must update their plans when users interrupt mid-sentence. Existing spoken-dialog tests mostly use tidy turns and miss this failure mode. EchoChain injects interruptions at a fixed point in assistant speech and names three recurring faults: contextual inertia, interruption amnesia and objective displacement. In a paired control without interruptions, failures fell by about 40 percent, and no tested real-time voice model passed more than half the cases, leaving large room for improvement.

Catalog record

CREATORSmit Nautambhai Modi, Gandharv Mahajan, Marc Wetter, Randall Welles
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE74358
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.16456

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

EchoChain: A Full-Duplex Benchmark for State-Update Reasoning Under Interruptions Smit Nautambhai Modi, Gandharv Mahajan, Marc Wetter, Randall Welles Abstract Real-time voice assistants must revise task state when users interrupt mid-response, but existing spoken-dialog benchmarks largely evaluate turn-based interaction and miss this failure mode. We introduce EchoChain, a controlled benchmark for evaluating full-duplex state-update reasoning under mid-speech interruptions. EchoChain identifies three recurring failure patterns in post-interruption continuations: contextual inertia, interruption amnesia, and objective displacement. The benchmark generates scenario-driven conversations and injects interruptions at a standardized point relative to assistant speech onset, enabling controlled cross-model comparison. In a paired half-duplex control, total failures drop by 40.2% relative to interrupted runs, indicating that many errors are driven by state-update reasoning under interruption rather than task difficulty alone. Across evaluated real-time voice models, no system exceeds a 50% pass rate, showing substantial room for improvement in mid-generation state revision. EchoChain provides a reproducible benchmark for diagnosing state-update reasoning failures in full-duplex voice interaction. 1 Introduction Human communication is naturally full-duplex. Listeners interject, revise, and provide information while the speaker is still talking through audio and visual cues ( 12 ) . Spoken dialogue systems are expected to operate under the same conditions, yet evaluation has largely remained half-duplex, assuming that each turn is entirely complete before the other party begins speaking ( 4 ) , ( 11 ) , ( 2 ) . In practice, a user can revise a request, add a constraint, or redirect the task while the assistant is mid-response, and the assistant must update its in-progress reasoning without the benefit of clean turn breaks. When new information arrives while the assistant is generating, online state revision becomes a core requirement, and existing benchmarks that assume turn boundaries do not evaluate it. Interruptions are a natural stressor for full-duplex systems because new information arrives while the assistant is mid-generation, forcing it to revise what it is currently producing ( 12 ) . The difficulty in evaluating this is that failures are often invisible at the surface. A model may acknowledge the interruption yet continue under stale or incorrect assumptions, incorporate the update momentarily before reverting to its original plan, or not complete the main requirements of the user input. The response appears fluent and responsive, but the underlying task state is incorrect. To assess whether these errors stem from the interruption itself rather than general task difficulty, we conducted a paired comparison against a non-interruption baseline of the same conversation using the same rubric. Under this baseline, with 48 sampled conversations across all 4 models, 92 failures were detected for the interrupted conversations, while 55 failures were detected for the non-interrupted conversations. This corresponds to a net 40.22% reduction in total failures, indicating that a substantial share of errors are related to the interruption event. This finding necessitates the creation of a dedicated benchmark on how models revise in-progress responses as new information arrives during generation. State-update reasoning is the ability of a model to preserve and revise its task state. In this paper, we focus specifically on when this happens during ongoing speech generation because a user introduces new information mid-utterance. Analysis of these failures reveals three recurring patterns in how models mishandle interruptions during generation, which we use to build a classification taxonomy. In contextual inertia, the model receives the new information but continues generating as if nothing changed. In interruption amnesia, the update is initially incorporated but later overwritten as generation proceeds. In objective displacement, the model pivots entirely to the interruption, abandoning the original task. Each pattern reflects a different breakdown in how the model maintains and revises task state under concurrent input. To study these failures systematically, we introduce EchoChain, a benchmark evaluating how full-duplex spoken dialogue models handle mid-generation interruptions. The EchoChain framework generates scenario-driven conversations through a planner-simulator loop, where each user message and barge-in is conditioned on the conversation history, ensuring that interruption content is contextually grounded. User speech and the interruption stimulus are pre-synthesized, and barge-in timing is anchored, so the framework can be used to compare various models under the same conditions. All user audio is generated through a Persona Voice Engine that clones voices from our diverse set of recorded actors into synthetic speech, ensuring identical audio input across models and controlled reproducibility across trials. A paired listening study shows that raters cannot reliably tell the cloned voices apart from real recordings. The framework records aligned audio and transcripts for the pre-interruption assistant segment, the barge-in utterance, and the post-interruption continuation, and routes flagged cases through automated classification followed by human review. 2 Failure Taxonomy Figure 1 : Illustrative examples of the three post-interruption failure modes in full-duplex state-update reasoning. Left: ( Contextual Inertia ) The assistant acknowledges the interruption but fails to correctly revise its prior reasoning. Center: ( Interruption Amnesia ) Previously established constraints are forgotten after the interruption. Right: ( Objective Displacement ) The assistant responds only to the interruption and fails to complete the original request. We categorize full-duplex state-update reasoning failures along three dimensions. These categories capture common post-interruption failure patterns that arise as a consequence of the interruption. Examples of these failures can be found in Figure 1 . Contextual Inertia is a failure that is seen when the assistant does not properly integrate the interruption content into the following response. Oftentimes, it does acknowledge that some interruption happened, even saying it is going to incorporate it; however, in the actual content, no change or an incorrect change is made instead. As in the example, the assistant acknowledges a caffeine constraint from the interruption but still recommends a caffeinated drink. This results in responses where it can appear to shift topic or correct itself superficially, but the actual content remains a failure. To date, established metrics remain insufficient for capturing this error, as they were not developed with this specific vulnerability in mind. Interruption Amnesia is a symptom of models simply forgetting past constraints or requirements after being interrupted. It can be seen that initially, constraints are incorporated correctly into earlier model responses; however, due to an onset of ’amnesia’ caused by the interruption, they fail to integrate them correctly after the barge-in. In the example, the assistant correctly applies dual-timezone formatting after the first interruption, but silently drops it after the second. As a result, the system loses contextual alignment in its response once the interruption occurs and generates a failing response. Objective Displacement is a unique failure that is caused by how an interruption changes a natural conversation. When the assistant is interrupted before it has completed a user request, in some cases, it is more than reasonable to expect the assistant to finish the request despite a change or modification to the request from the user. This is signaled by the interruption either adding context without introducing a new request, or using additive phrasing such as ’also’ or ’in addition to.’ Yet, it can be observed that most of the time, the assistant responds only to the interruption, entirely forgetting the actual conversation state. The example shows the assistant abandoning a regulatory comparison request when asked for an additional flag in addition to the comparison request. This leaves the conversation broken

Read the complete text