A Universal Avoidance Method for Diverse Multi-branch Generation
Kyeongman Park, Minha Jhang, Kyomin Jung · 2026
Read A Universal Avoidance Method for Diverse Multi-branch Generation by Kyeongman Park, Minha Jhang, Kyomin Jung (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-17323.txt.
Introduction
Park, Jhang, and Jung propose UAG, a simple rule for coaxing varied outputs from generative models: penalize each new branch for resembling earlier ones. The trick works for both diffusion and transformer models with little extra computation. Experiments report nearly twice the diversity at over four times the speed, using only a tiny fraction of the rival method's computing effort. The aim is creativity closer to human variety without heavy machinery.
Catalog record
| CREATOR | Kyeongman Park, Minha Jhang, Kyomin Jung |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38502 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.17323
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Universal Avoidance Method for Diverse Multi-branch Generation Kyeongman Park, Minha Jhang, Kyomin Jung Abstract Modern generative models still lack human-level creativity, particularly in multi-branch diversity. Prior approaches to address this problem often incur heavy computation or strong dependency on model architecture. Therefore, we introduce UAG ( U niversal A voidance G eneration), a model-agnostic and computationally efficient generation strategy that penalizes similarity among previously generated outputs. Thus, UAG can enhance multi-branch diversity across both diffusion and transformer models, with minimal additional computation. In experiments, our method achieves up to 1.9 times higher diversity, runs 4.4 times faster, and requires only 1/64 of the FLOPs compared to state-of-the-art methods. The full code is here . 1 Introduction Recently, generative models across various domains such as text and image generation 28 ; 4 ; 15 ; 18 ; 6 ; 17 ; 24 have demonstrated remarkable performance. Despite this progress, these models still fall short in terms of human-level creativity, particularly in tasks such as story writing 20 ; 27 ; 16 ; 11 and illustration 14 ; 12 ; 29 . These tasks require substantial multi-branch diversity , i.e., generating diverse outputs from the same prompts 22 . Thus, many approaches have been proposed to address the lack of multi-branch diversity in these tasks 22 ; 16 ; 27 ; 14 ; 12 . However, these approaches often suffer from substantial computational overhead due to token-wise or pixel-wise operations 22 ; 14 , and limited applicability across different models because of their dependence on specific architectures 22 ; 2 ; 26 ; 12 ; 23 . To overcome these limitations, we propose UAG ( U niversal A voidance G eneration), a novel framework for enhancing multi-branch diversity that minimizes additional computation costs while remaining agnostic to model architecture. UAG enhances multi-branch diversity by reducing similarity to previously generated outputs, by penalizing the gradient of a similarity loss on the final outputs. The similarity loss comprises two components:(1) a local similarity loss at the token or pixel level, and(2) a global similarity loss at the hidden-state level. Through loss scheduling, UAG can promote a natural progression of diversity: in the early stages, local similarity encourages concept-level diversity, while in later stages, global similarity fosters semantic-level diversity. Because UAG requires only a few gradient computations per sample, rather than token-wise 22 ; 26 ; 7 ; 25 ; 3 ; 8 or pixel-wise operations 10 , it is both highly efficient and fast. Moreover, due to its simplicity, UAG is model-agnostic and thus applicable to both diffusions and transformers, which represent the current state of the art in generative modeling. Surprisingly, our method achieves on average 1.43 times higher scores in conventional diversity metrics(e.g., BLEU, CLIPscore) and 1.19 times higher scores in LLM-based evaluations compared to baselines. Moreover, our method simultaneously attains 4.4 times faster decoding speed and requires 64 times fewer FLOPs than previous state-of-the-art methods 22 ; 14 . 2 Universal Generation Process Most modern generative models, including autoregressive and diffusion-based models, can be uniformly described as sequential processes indexed by steps t=1,2,\dots,T . At each step t , the model updates an internal hidden(or latent) state h_{t} and produces an output representation y_{t} : (y_{t},h_{t+1})\sim P_{\theta}(\cdot\mid h_{t},p), (1) where p is the conditioning input(e.g., a text prompt or image condition). We define this process as the Universal Generation Process, and this makes it possible to design universal diverse multi-branch generation strategy that operates consistently across different model families. Autoregressive language models. The autoregressive language models include famous LLM such as GPT-5 18 , LLaMa-3B 4 , and Qwen-7B 28 . For these models, h_{t} denotes the decoder’s last hidden state after processing tokens up to step t , while the model output y_{t} is the logit vector obtained by applying the output weight matrix to h_{t} . The next state h_{t+1} is updated once a token is predicted and appended. Diffusion language models. For denoising diffusion language models 30 ; 15 , the process starts from random noise and iteratively refines it into a coherent text representation, where the steps t typically count down from a maximum value T to 1. Here, the state h_{t} is the noisy latent representation of the text at reverse-time step t . The model output y_{t} is the prediction of the token logits. The next state in the generative sequence, h_{t-1} , is then calculated by a scheduler, which uses the current state h_{t} and the prediction y_{t} to produce a slightly less noisy representation. Diffusion image models. Similarly, latent diffusion models for image generation 9 ; 24 also follow a reverse, denoising process within the VAE’s compressed latent space. The state h_{t} is the noisy latent vector at reverse-time step t . The model’s output y_{t} is the predicted noise present in h_{t} . The scheduler uses this predicted noise y_{t} to remove it from the current state h_{t} , thereby computing the next, cleaner state h_{t-1} . This process is repeated until the final denoised latent h_{0} is obtained, which is then decoded by the VAE into the final image. 3 Universal Avoidance Generation To achieve diverse multi-branch generation within the Universal Generation Process, we propose UAG ( U niversal A voidance G eneration), which applies gradient-based penalties to the output space and adjusts the results step by step. Our method requires only simple differentiation with respect to outputs and hidden states, rather than repetitive token-wise or pixel-wise computation. Thus, it is not only applicable to any generative model within the Universal Generation Process, but also significantly more efficient than previous state-of-the-art methods 22 ; 14 . 3.1 Step-wise Loss Scheduling We denote by \phi(y_{t}) the output representation of the model. Let \mathcal{B}_{t}^{\text{out}} and \mathcal{B}_{t}^{\text{hid}} be the reference banks, which are previously generated outputs or hidden states from cached past runs. The step- t local loss \mathcal{L}^{(t)}_{\text{local}} is defined as \max_{b\in\mathcal{B}^{\text{out}}_{t}}\,\mathrm{sim}\!\big(\phi(y_{t}),\,b\big) , and global loss \mathcal{L}^{(t)}_{\text{global}} is defined as \max_{b\in\mathcal{B}^{\text{hid}}_{t}}\,\mathrm{sim}\!\big(h_{t},\,b\big) , where \mathrm{sim}(\cdot,\cdot) is a similarity measure that depends on the model type(see Appendix D ). To balance the contributions of the two losses, we apply a logistic schedule, following prior work 22 : s_{t}=\frac{1}{1+e^{\delta\,(t-L_{0})}},\\ w_{\text{local}}(t)=\alpha\,\cdot s_{t},\quad w_{\text{global}}(t)=\beta\,\cdot(1-s_{t}) (2) where \alpha,\beta are maximum weights, L_{0} is the transition center, and \delta controls sharpness. Thus, we emphasize local similarity(e.g., token-level or pixel-level) in the early stages, while in the later stages we emphasize global similarity(e.g., hidden-state level). This scheduling is intuitive, since in the earlier stages the model has not yet formed meaningful global representations and should therefore focus on local contextual diversity, whereas in the later stages it becomes more important to ensure global semantic diversity(e.g., story narratives or scene layouts). Finally, we define the UAG loss at step t as a combined loss as follows: \mathcal{L}^{(t)}_{\text{UAG}}=w_{\text{local}}(t)\,\mathcal{L}^{(t)}_{\text{local}}+w_{\text{global}}(t)\,\mathcal{L}^{(t)}_{\text{global}}. (3) 3.2 Gradient-based Penalty Adaptation To penalize similarity to previous outputs, we adjust the output representation y_{t} by subtracting the gradient of the UAG loss: \hat{y}_{t}=y_{t}-\nabla_{y_{t}}{L}^{(t)}_{\text{UAG}}\\ =y_{t}-\Big(w_{\text{local}}(t)\,\nabla_{y_{t}}\mathcal{L}^{(t)}_{\text{local}}+w_{\text{global}}(t)\,\nabla_{y_{t}}\mathcal{L}^{(t)}_{\text{global}}\Big) (4) then we use \hat{y}_{t} to sample token probabilities or as input for the scheduler of the diffusion model. This penalty introduces a repulsive force that pushes the current generation away from previously generated outputs. For the underlying mathematical motivations of this adjustment, see Appendix B .