DiscreteRTC: Discrete Diffusion Policies are Natural Asynchronous Executors

Pengcheng Wang, Kaiwen Hong, Chensheng Peng, Katherine Driggs-Campbell, Masayoshi Tomizuka, Chenfeng Xu, Chen Tang, UC Berkeley, UIUC, UT Austin, UCLA [email protected], [email protected], [email protected] · 2026

Read DiscreteRTC: Discrete Diffusion Policies are Natural Asynchronous Executors by Pengcheng Wang, Kaiwen Hong, Chensheng Peng, Katherine Driggs-Campbell, Masayoshi Tomizuka, Chenfeng Xu, Chen Tang, UC Berkeley, UIUC, UT Austin, UCLA [email protected], [email protected], [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-25050.txt.

Introduction

Wang and six colleagues propose DiscreteRTC, discrete diffusion policies as natural async executors. Discrete diffusion policies make actions by unmasking. Inpainting is native to them, so no fine-tuning is needed, and early stopping adds adaptive help. DiscreteRTC replaces outside corrections with native unmasking. On dynamic simulated benchmarks and real handling tasks, it beats continuous chunking and other baselines. It needs no extra code. It uses about 0.7 times the compute of fresh generation. It raises success by 65 percent in a real-world hockey defend task.

Catalog record

CREATORPengcheng Wang, Kaiwen Hong, Chensheng Peng, Katherine Driggs-Campbell, Masayoshi Tomizuka, Chenfeng Xu, Chen Tang, UC Berkeley, UIUC, UT Austin, UCLA [email protected], [email protected], [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41816
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.25050

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

DiscreteRTC: Discrete Diffusion Policies are Natural Asynchronous Executors Pengcheng Wang, Kaiwen Hong, Chensheng Peng, Katherine Driggs-Campbell, Masayoshi Tomizuka, Chenfeng Xu, Chen Tang, UC Berkeley, UIUC, UT Austin, UCLA [email protected], [email protected], [email protected] Abstract Unlike chatbots, physical AI must act while the world keeps evolving. Therefore, the inter-chunk pause of synchronous executors are fatal for dynamic tasks regardless of how fast the inference is. Asynchronous execution— thinking while acting —is therefore a structural requirement, and real-time chunking (RTC) makes it viable by recasting chunk transitions as inpainting: freezing committed actions and consistently generating the remainder. However, RTC with flow-matching policy is structurally suboptimal: its inpainting comes from inference-time corrections rather than the base policy, yielding little pre-training benefit, specific fine-tuning, heuristic guidance, and extra computation that inflates the latency. In this work, we observe that discrete diffusion policies, which generate actions by iteratively unmasking, are natural asynchronous executors that resolve all limitations at once : they are fine-tuning free since inpainting is their native operation, while early stopping further provides adaptive guidance and reduces inference cost. We propose DiscreteRTC , which replaces external corrections with native unmasking, and show on dynamic simulated benchmarks and real-world dynamic manipulation tasks that it achieves higher success rates than continuous RTC and other baselines. In summary, DiscreteRTC is simpler to implement with 0 lines of code for async inpainting, faster at inference with only ∼ 0.7 × \sim 0.7\times computation compared with generating actions from scratch, and better at execution with 50 % \% higher success rate in real-world dynamic pick task compared with flow-matching-based RTC. More visualizations are on our website . Figure 1: Async Execution with discrete diffusion policies solving dynamic manipulation. Gray rectangles and blocks represent the action chunks and the actions. Yellow and green cubes represent the masked and unmasked action tokens. During each inference cycle, discrete diffusion policies copy the tail of the last action chunk as the committed prefix, and inpaint upon it by simply forwardin itself. Compared with flow-matching-based inpainting that relies on \Pi GDM, discrete-diffusion-based inpainting inference is simper to implement, faster at inference, and better at execution. 1 Introduction Unlike chatbots [ 7 , 44 ] or image generators [ 37 ] , physical AI systems [ 3 , 15 , 2 , 39 , 28 , 1 , 34 , 47 ] act while the world keeps evolving: the robot cannot afford to pause while thinking. This reveals a structural limitation of synchronous execution with action-chunking policies [ 10 , 18 ] —every inference call forces a mandatory pause between chunks during which the robot is idle but the world is not, which is fatal for dynamic tasks such as tracking moving targets [ 43 ] . Therefore, asynchronous execution— thinking while acting over action chunks—is not a latency optimization but a structural requirement. However, naive async strategies, whether switching chunks immediately or smoothing them via temporal ensembling [ 49 ] , introduce inter-chunk discontinuity or invalid actions because each new chunk is generated in isolation from the committed actions. Real-time chunking (RTC) [ 4 ] resolves this by recasting chunk transitions as an inpainting problem: freezing committed actions from the previous chunk and consistently generating the remainder, representing a simple yet effective approach that has achieved great success in precise and smooth real-world manipulation [ 14 , 13 ] . Specifically, RTC is specially designed for flow-matching policies [ 20 ] . As the predominant action head architecture in today’s state-of-the-art VLAs [ 3 , 15 , 2 ] , flow-matching has driven much of the recent progress in large-scale robot learning. However, in this paper, we first systematically show that RTC with a flow-matching head is far from ideal with four critical limitations : (a) Pre-training without Inpainting. Flow-matching policies are not pre-trained on action chunks with inconsistent noise levels for inference-time inpainting. Therefore, scaling pre-training does not directly improve asynchronous performance, and inference-time corrections like \Pi GDM [ 32 ] are inevitable for effective inpainting; (b) Fine-tuning Required. As a consequence, adequate inpainting quality demands a dedicated fine-tuning stage with techniques such as action-suffix conditioning [ 5 , 38 ] to explicitly introduce the inpainting-specific noise pattern into training; (c) Heuristic Guidance. Moreover, to better leverage the previous action chunks, \Pi GDM requires a heuristic schedule fixed across different inference-time cases; (d) Extra Inference Cost. Finally, the correction guidance term at every denoising step roughly doubles inference cost, ironically increasing the very latency RTC aims to hide. Many ongoing efforts [ 23 , 25 , 5 , 45 ] seek to resolve these aforementioned issues within the flow-matching paradigm. Instead, our key insight and observation are that by replacing the action head with a discrete diffusion policy [ 19 ] , all the aforementioned limitations can be resolved at once. Or, to put it simply: Discrete Diffusion Policies are Natural Asynchronous Executors : (a) Inpainting as Pre-training. Discrete diffusion policies are pre-trained to inpaint upon randomly masked sequences. Therefore, scaling pre-training directly improves asynchronous performance, and the native forward pass suits inference-time inpainting; (b) Fine-tuning Free. As a consequence, inpainting-specific patterns are implicitly introduced during pre-training, making discrete diffusion a fine-tuning-free approach for high-quality, out-of-the-box asynchronous execution; (c) Natural Guidance. Moreover, with discrete diffusion policies, we can early-exit inference once the necessary action tokens are unmasked, leaving the remaining masking pattern as an adaptive and natural guidance for the next inference; (d) Lower Inference Cost. Finally, with committed tokens from previous chunks, the tokens to unmask per inference is reduced, leading to lower inference cost for inpainting. We propose the resulting async execution method, DiscreteRTC , which replaces the external correctiosn in flow-matching-based RTC with the native unmasking operation of discrete diffusion policies. Experiments on the dynamic simulated benchmark Kinetix [ 27 ] and real-world dynamic manipulation demonstrate that DiscreteRTC achieves higher success rates than flow-matching-based RTC and other baselines. In summary, DiscreteRTC is simpler to implement with 0 lines of code for async inpainting, faster at inference with only \ 0.7\times computation compared with generating actions from scratch while flow-matching-based RTC requires \sim 1.7\times computation, and better at execution with 50 \% higher success rate in real-world dynamic pick over flow-matching-based RTC. 2 Preliminaries 2.1 Real-time Chunking Action Chunking Flow Policy. We consider an action chunking policy \pi that receives an observation \mathbf{o}_{t} and outputs a chunk of H future actions \mathbf{A}_{t}=(a_{t},a_{t+1},\ldots,a_{t+H-1}) . State-of-the-art VLAs [ 3 , 15 ] generate action chunks using conditional flow matching [ 20 , 48 ] : noise \mathbf{A}^{0}_{t}\sim\mathcal{N}(0,I) is sampled and integrated through the learned velocity field v_{\pi} from \tau=0 to 1 : \mathbf{A}_{t}^{\tau+\frac{1}{n}}=\mathbf{A}_{t}^{\tau}+\frac{1}{n}v_{\pi}(\mathbf{A}_{t}^{\tau},\mathbf{o}_{t},\tau), (1) where \tau\in[0,1) is the flow timestep and n is the number of denoising steps. During training, all actions within a chunk are corrupted at the same noise level \tau and denoised together. Real-Time Chunking. Running modern VLAs with action generation time \delta smaller than the controller period \Delta t (typically 20 \sim 50 ms) is nearly impossible [ 4 ] : even a 7B OpenVLA model optimized for speed [ 16 , 17 ] requires \delta=321 ms on a server-grade A100 GPU. The robot must therefore act while the policy is still computing, i.e. , asynchronous execution. Naively switching to

Read the complete text