PairFlow: Closed-Form Source-Target Coupling for Few-Step Generation in Discrete Flow Models

Mingue Park Jisung Hwang Seungwoo Yoo Kyeongmin Yeo Minhyuk SungKAIST {kicikicik,4011hjs,dreamy1534,aaaaa,mhsung}@kaist.ac.kr · 2025

Read PairFlow: Closed-Form Source-Target Coupling for Few-Step Generation in Discrete Flow Models by Mingue Park Jisung Hwang Seungwoo Yoo Kyeongmin Yeo Minhyuk SungKAIST {kicikicik,4011hjs,dreamy1534,aaaaa,mhsung}@kaist.ac.kr (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-20063.txt.

Introduction

Mingue Park and four colleagues accelerate discrete flow models, a newer family of generators for discrete data like molecules and token images. Such models sample iteratively and slowly, and existing speedups lean on fine-tuning with its extra training cost. PairFlow instead intervenes before training, as a lightweight preprocessing step needing no pretrained teacher.

Inspired by ReFlow, the method trains on coupled source and target samples, and its engine is a closed-form inversion for discrete flow models that constructs those pairs efficiently. The preprocessing costs under two percent of full training compute, yet matches or beats two-stage fine-tuning pipelines, and the resulting models distill even better afterward. Experiments span molecular data plus binary and color images, and the work appears at ICLR 2026.

Catalog record

CREATORMingue Park Jisung Hwang Seungwoo Yoo Kyeongmin Yeo Minhyuk SungKAIST {kicikicik,4011hjs,dreamy1534,aaaaa,mhsung}@kaist.ac.kr
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE86 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-20063

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

PairFlow: Closed-Form Source-Target Coupling for Few-Step Generation in Discrete Flow Models Mingue Park Jisung Hwang Seungwoo Yoo Kyeongmin Yeo Minhyuk SungKAIST {kicikicik,4011hjs,dreamy1534,aaaaa,mhsung}@kaist.ac.kr Abstract We introduce PairFlow , a lightweight preprocessing step for training Discrete Flow Models (DFMs) to achieve few-step sampling without requiring a pretrained teacher. DFMs have recently emerged as a new class of generative models for discrete data, offering strong performance. However, they suffer from slow sampling due to their iterative nature. Existing acceleration methods largely depend on finetuning, which introduces substantial additional training overhead. PairFlow addresses this issue with a lightweight preprocessing step. Inspired by ReFlow and its extension to DFMs, we train DFMs from coupled samples of source and target distributions, without requiring any pretrained teacher. At the core of our approach is a closed-form inversion for DFMs, which allows efficient construction of paired source–target samples. Despite its extremely low cost, taking only up to 1.7% of the compute needed for full model training, PairFlow matches or even surpasses the performance of two-stage training involving finetuning. Furthermore, models trained with our framework provide stronger base models for subsequent distillation, yielding further acceleration after finetuning. Experiments on molecular data as well as binary and RGB images demonstrate the broad applicability and effectiveness of our approach. † † footnotetext: \ast Equal contribution. 1 Introduction Discrete Flow Models (DFMs) ( 4 ; 8 ) have recently emerged as a promising class of generative models, extending the idea of Flow Models (FMs) ( ) for continuous data to the discrete domain. By adapting flow-based principles to categorical structures, DFMs provide a principled and efficient way to capture complex discrete distributions through iterative sampling. They have shown success across a variety of applications, particularly in scientific domains such as molecule generation ( 22 ; 11 ) , where DFMs offer a natural framework for modeling chemical structures and generating novel candidates. Analogous to FMs in the continuous domain, a key challenge of DFMs is the long computation time for generation due to their iterative sampling nature. Recent work ( 7 ; 9 ; 24 ; 33 ) have sought to accelerate the generative process through distillation-based finetuning, which builds on ideas originally developed for continuous flow matching. Notably, ReFlow ( 32 ) is a well-known technique for FMs that pairs samples from the source (prior) distribution and the target distribution by running the generative process of a pretrained model and using the resulting pairs for finetuning. Recently, this idea has also been extended to DFMs ( 33 ) to reduce conditional total correlation through finetuning, thereby enabling few-step generation. Despite these promising results in acceleration, distillation-based methods incur substantial finetuning overhead, amounting to about 10–20% of the time required to train the base model from scratch. In other words, the gain in generation speed comes at the expense of considerable additional training cost. To our knowledge, no prior work has addressed this training-time cost when pursuing inference-time acceleration. This raises a natural question: can we achieve speedups comparable to distillation-based approaches while requiring only a lightweight preprocessing phase that requires orders of magnitude less compute, on the order of tens of GPU minutes? We propose PairFlow , a training framework for DFMs that enables few-step sampling by constructing paired source–target samples using closed-form velocities. While inspired by ReDi-style coupling-driven training, our approach eliminates the need for a pretrained teacher by using closed-form formulations and achieves acceleration without finetuning. The algorithm for computing source–target pairs is fully parallelizable and requires at most 1.7% of compute needed for full model training. Despite relying only on a lightweight preprocessing step, PairFlow attains performance comparable to, or even superior to state-of-the-art distillation-based techniques, which can require up to 143 times more computation. Furthermore, models trained with our technique provide stronger bases for subsequent distillation, delivering additional performance gains while incurring only minimal preprocessing cost. At the core of our framework is the simulation of probability paths connecting source (prior) and target (data) distributions in discrete spaces, made possible by closed-form expressions of velocities. While closed-form forward velocities have been studied for flow models in continuous domains ( 12 ; 3 ) , they have, to the best of our knowledge, neither been explored for DFMs nor applied to identifying suitable source–target pairs in the context of distillation-based acceleration, as in ReDi ( 33 ) . In this work, we investigate this idea for the first time. For DFMs, with a particular focus on uniform-state models ( 24 ; 26 ) equipped with a self-correcting mechanism, we show that the closed-form forward velocity is determined by the Hamming distance, which measures the number of differing tokens between two sequences. Using this velocity, samples from the source (latent) distribution can be mapped to given target samples. However, because multiple source samples may map to the same target, covering all targets through coupling would require an impractically large number of source samples. To overcome this, we derive the corresponding backward velocity in closed form and leverage it to simulate backward probability paths that efficiently map data points to source points, making pair discovery computationally efficient. In our experiments, we show that the proposed framework enables few-step sampling across diverse discrete domains, including molecular data ( 22 ; 11 ) and 2D images, exemplified by MNIST-Binary ( 16 ) and CIFAR-10 ( 15 ) . On the QM9 ( 22 ) and ZINC-250k ( 11 ) datasets, PairFlow not only improves the base model but also performs comparably to, or even better than, distilled models that require up to 143\times more compute during finetuning, compared to our lightweight preprocessing algorithm. Similar improvements are observed on MNIST-Binary, where models paired with PairFlow achieve performance comparable to those using DCD ( 24 ) and ReDi ( 33 ) , while being up to 35\times faster. Furthermore, after subsequent distillation, base models trained with pairs generated by our method consistently achieve higher performance, underscoring the importance of well-constructed source–target pairings. 2 Related Work 2.1 Discrete Flow models The concept of Flow Matching ( 18 ) has recently been extended to the discrete flow-based models ( 8 ; 4 ; 23 ; 26 ) , demonstrating its flexibility across high-dimensional and structured data ( 2 ; 5 ; 30 ; 1 ; 19 ; 34 ; 17 ; 4 ; 29 ) . Among these, uniform-state models ( 24 ; 26 ) have been studied for their self-correcting properties, which enable recovery from errors introduced during parallel decoding. However, their performance degrades markedly in few-step settings, posing a key limitation for efficient generation under tight compute budgets. 2.2 Accelerating Discrete Flow models Several approaches have been proposed to accelerate sampling with DFMs. 21 directly optimize sampling timesteps for improved parallelism while mitigating decoding errors. 9 highlight the importance of capturing dimensional correlations for faster sampling and introduce mixture models to this end, at the cost of additional loss terms that complicate optimization. Another line of work adapts techniques from continuous domains, as in 24 , that propose a discrete analogue of Consistency Distillation (CD) ( 27 ) by leveraging the duality between uniform-state and continuous Gaussian models. Most relevant to our approach is ReDi ( 33 ) , which draws inspiration from the

Read the complete text