Be More Diverse than the Most Diverse: Optimal Mixtures of Generative Models via Mixture-UCB Bandit Algorithms

Parham Rezaei, Farzan Farnia, Cheuk Ting Li · 2024

Read Be More Diverse than the Most Diverse: Optimal Mixtures of Generative Models via Mixture-UCB Bandit Algorithms by Parham Rezaei, Farzan Farnia, Cheuk Ting Li (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-17622.txt.

Introduction

Parham Rezaei, Farzan Farnia, and Cheuk Ting Li show that mixing generative models beats picking the best one in this 2024 paper. Labs now hold many trained generators, and the usual move is to keep the top scorer. On image benchmarks, the authors found a blend of models scored better on quality and diversity measures than any single model in it. They then asked how to find the best blend weights with few test samples. The search is framed as a quadratic optimization over kernel based scores.

They recast that search as a bandit problem and propose an algorithm, Mixture-UCB. It provably settles on the optimal mixture. A regret bound supports the claim, and tests on text and image generators show it working. The method extends to any convex quadratic goal over mixture weights. The authors argue that model zoos should be blended, not culled.

Catalog record

CREATORParham Rezaei, Farzan Farnia, Cheuk Ting Li
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE81 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-17622

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Be More Diverse than the Most Diverse: Optimal Mixtures of Generative Models via Mixture-UCB Bandit Algorithms Parham Rezaei, Farzan Farnia, Cheuk Ting Li Abstract The availability of multiple training algorithms and architectures for generative models requires a selection mechanism to form a single model over a group of well-trained generation models. The selection task is commonly addressed by identifying the model that maximizes an evaluation score based on the diversity and quality of the generated data. However, such a best-model identification approach overlooks the possibility that a mixture of available models can outperform each individual model. In this work, we numerically show that a mixture of generative models on benchmark image datasets can indeed achieve a better evaluation score (based on FID and KID scores), compared to the individual models. This observation motivates the development of efficient algorithms for selecting the optimal mixture of the models. To address this, we formulate a quadratic optimization problem to find an optimal mixture model achieving the maximum of kernel-based evaluation scores including kernel inception distance (KID) and Rényi kernel entropy (RKE). To identify the optimal mixture of the models using the fewest possible sample queries, we view the selection task as a multi-armed bandit (MAB) problem and propose the Mixture Upper Confidence Bound (Mixture-UCB) algorithm that provably converges to the optimal mixture of the involved models. More broadly, the proposed Mixture-UCB can be extended to optimize every convex quadratic function of the mixture weights in a general MAB setting. We prove a regret bound for the Mixture-UCB algorithm and perform several numerical experiments to show the success of Mixture-UCB in finding the optimal mixture of text and image generative models. The project code is available at https://github.com/Rezaei-Parham/Mixture-UCB . 1 Introduction The rapid advancements in generative modeling have created a need for mechanisms to combine multiple well-trained generative models, each developed using different algorithms and architectures, into a single unified model. Consider m unconditional generative models \mathcal{G}_{1},\ldots,\mathcal{G}_{m} , where each \mathcal{G}_{i} represents a probability model P_{\mathcal{G}_{i}} according to which new samples are generated. A common approach for creating a unified model is to compute evaluation scores (e.g., the standard FID ( Heusel et al., 2017 ) and KID ( Bińkowski et al., 2018 ) scores) that quantify the diversity and fidelity of the generated data, followed by selecting the model P_{\mathcal{G}_{i^{*}}} with the best evaluated score. This best-score model selection strategy has been widely adopted for choosing generative models across various domains, including image, text, and video data generation. Figure 1: A mixture (right-most case) of FFHQ pre-trained generative models with weights (0.25,0.4,0.01,0.08,0.26) achieves better FID and KID scores compared to each of the five involved models. The mixture weights are computed using our proposed Mixture-UCB-OGD algorithm. However, the model selection approach by identifying the score-maximizing model overlooks the possibility that a mixture of the generative models \alpha_{1}P_{\mathcal{G}_{1}}+\cdots+\alpha_{m}P_{\mathcal{G}_{m}} , where each sample is generated from a randomly-selected model with \alpha_{i} being the probability of selecting model \mathcal{G}_{i} , can outperform every individual model. This motivates the following question: Can there be real-world settings where a non-degenerate mixture of some well-trained generative models obtain a better evaluation score compared to each individual model? Note that the standard FID and KID scores are convex functions of the generative model’s distribution, and thus they can be optimized by a non-degenerate mixture of the models. In this work, we numerically show that it is possible for a mixture of real-world generative models to improve evaluation scores over the individual models. An example is shown in Figure 1 , where we find a non-degenerate mixture of five generative models pre-trained on the FFHQ dataset 1 1 1 The pre-trained generative models are downloaded from dgm-eval GitHub repository ( Stein et al., 2023 ) . . As shown, assigning mixture weights (0.25,0.4,0.01,0.08,0.26) to the models results in a significantly better FID score 170.11 than the best individual FID 185.07 . To understand the improvement achieved by the mixture model, we note that the FID and KID scores evaluate both the quality and diversity of the generated data. While the averaged quality of generated samples represents an expected value that is optimized by an individual model, the diversity of samples from a mixture of the models can significantly improve over the diversity of the individual models’ data. Figure 2 displays an illustrative example for this point, where we observe that the diversity of ‘‘red bird, cartoon style’’ samples generated by each of the three text-to-image models, is qualitatively and quantitatively 2 2 2 We have evaluated the diversity scores RKE ( Jalali et al., 2023 ) and Vendi ( Dan Friedman & Dieng, 2023 ) . lower than the diversity of their mixture. As a result, the improvement in the diversity of a mixture of generative models can result in an improved FID and KID evaluation scores, as we numerically observe in Figure 1 . 1.1 Computing Optimal Mixtures of Generative Models: The Mixture-UCB Multi-Armed Bandit Algorithm Since the evaluation score of a mixture of generative models can improve over the scores of the individual models, a natural question is how to efficiently compute the weights of an optimal mixture of the models using the fewest possible samples from the models. Here, our goal is to minimize the number of sample generation queries from sub-optimal models, which will save the time and monetary costs of identifying the best model. To achieve this, we propose viewing the task as a multi-armed bandit (MAB) problem, in which every generative model represents an arm and our goal is to find the best mixture of the models with the optimal evaluation score. The MAB approach for selecting among generative models has been recently explored by Hu et al. (2024) applying the Upper Confidence Bound (UCB) algorithm to the FID score. This MAB-based model selection extends the online model selection methods for supervised models, including the successive halving strategy ( Karnin et al., 2013 ; Jamieson & Talwalkar, 2016 ; Chen & Ghosh, 2024 ) 3 3 3 Jamieson & Talwalkar (2016) focused on applying successive halving on hyperparameter optimization for supervised learning, whereas Chen & Ghosh (2024) focused on generative models using maximum mean discrepancy ( Gretton et al., 2012 ) as the score. . However, in the existing MAB algorithms, the goal is to eventually converge to a single arm with the optimal score. Successive halving ( Karnin et al., 2013 ; Jamieson & Talwalkar, 2016 ; Chen & Ghosh, 2024 ) and the UCB algorithm developed by Hu et al. (2024) will ultimately select only one generative model after a sufficient number of iterations. However, as we discussed earlier, the evaluation scores of the generated data could be higher when the sample generation follows a mixture of models rather than a single model. This observation leads to the following task: Developing an MAB algorithm that finds the optimal mixture of the arms rather than the single best arm. In this work, we develop a general MAB method, which we call the Mixture-UCB algorithm, that can provably find the best mixture of the arms when the evaluation score is a quadratic function of the arms’ distribution, i.e. when it represents the average of scores

Read the complete text