Warm Starting of CMA-ES for Contextual Optimization Problems
Yuta Sekino, Kento Uchida, Shinichi Shirakawa · 2025
Read Warm Starting of CMA-ES for Contextual Optimization Problems by Yuta Sekino, Kento Uchida, Shinichi Shirakawa (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-12555.txt.
Introduction
Yuta Sekino, Kento Uchida, and Shinichi Shirakawa improve a search algorithm for problems where the best answer depends on outside conditions, called context. Their 2025 paper works with CMA-ES, a well-known evolutionary search method. In contextual problems, a factory setting or design must be tuned for each new situation, not just once.
Older policy models guess the answer for each context but are limited by their form. The team's method instead reuses past search results: a statistical model learns from old contexts, then gives the search a warm start near the likely answer for a new one. Simulations show the approach optimizes more efficiently. Numerical simulations test how well the warm start works.
Catalog record
| CREATOR | Yuta Sekino, Kento Uchida, Shinichi Shirakawa |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 40 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-12555
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Warm Starting of CMA-ES for Contextual Optimization Problems Yuta Sekino, Kento Uchida, Shinichi Shirakawa Abstract Several practical applications of evolutionary computation possess objective functions that receive the design variables and externally given parameters. Such problems are termed contextual optimization problems. These problems require finding the optimal solutions corresponding to the given context vectors. Existing contextual optimization methods train a policy model to predict the optimal solution from context vectors. However, the performance of such models is limited by their representation ability. By contrast, warm starting methods have been used to initialize evolutionary algorithms on a given problem using the optimization results on similar problems. Because warm starting methods do not consider the context vectors, their performances can be improved on contextual optimization problems. Herein, we propose a covariance matrix adaptation evolution strategy with contextual warm starting (CMA-ES-CWS) to efficiently optimize the contextual optimization problem with a given context vector. The CMA-ES-CWS utilizes the optimization results of past context vectors to train the multivariate Gaussian process regression. Subsequently, the CMA-ES-CWS performs warm starting for a given context vector by initializing the search distribution using posterior distribution of the Gaussian process regression. The results of the numerical simulation suggest that CMA-ES-CWS outperforms the existing contextual optimization and warm starting methods. 1 Introduction In practical applications of evolutionary algorithms, the evaluation value of the given objective function is often determined using the design variables and externally given parameters. These optimization problems are referred to as contextual optimization problems and require finding the optimal solution corresponding to the given external parameter, called context vector. Contextual optimization problems arise in many applications, such as the optimization of the controller model of a robot for target motion [ 2 , 4 ] and nuclear fusion control using plasma states as context vectors [ 7 ] . For example, in the controller model of robot locomotion task [ 2 ] , the target locomotion speed and the locomotion direction are possible choices of the context vector, and the objective of optimization is to design a controller model that realizes the locomotion of the robot with the target speed. Various contextual optimization methods have been proposed to efficiently optimize these contextual optimization problems [ 1 , 3 , 14 ] . These methods solve contextual optimization problems by training the policy model that receives the context vector and predicts the corresponding optimal solution. To reduce the training cost of the policy model in contextual optimizations, several studies [ 3 , 5 ] have focused on the update rules of the covariance matrix adaptation evolution strategy (CMA-ES) [ 12 ] , which employs a multivariate Gaussian distribution as the sampling distribution and iteratively updates the distribution parameters. The contextual CMA-ES [ 3 ] extends the update rules of the CMA-ES to update the policy model. Notably, because many contextual optimization methods use linear models as the policy model, their ability to predict the optimal solution is limited. Meanwhile, warm starting methods have been used to start the optimization of evolutionary algorithms from a good state on a given problem. The warm starting methods initialize the evolutionary algorithm using the optimization results on similar problems [ 16 , 20 ] . For example, the warm starting CMA-ES (WS-CMA-ES) initializes the multivariate Gaussian distribution using the set of superior solutions achieved in the optimization of a given similar task. However, the existing warm starting methods do not consider the context vectors. We are of the view that the performance of warm starting methods on contextual optimization problems can be improved using the information of context vectors. A related idea was introduced in the contextual bandit problems [ 21 ] , which utilize a dataset consisting of a set of context vectors provided in advance by experts and a set of reward vectors for each action. In this study, we propose the CMA-ES with contextual warm starting (CMA-ES-CWS) to efficiently obtain the optimal solution corresponding to a given context vector. CMA-ES-CWS requires the best solutions corresponding to some context vectors achieved in past optimizations. Given the optimization results for past context vectors, CMA-ES-CWS trains the multi-output Gaussian process regression (GPR) to predict the optimal solution from the context vector. Next, CMA-ES-CWS performs warm starting for a newly obtained context vector by initializing the search distribution using the predictive distribution of the GPR. We evaluate the performance of the CMA-ES-CWS using the benchmark functions and control task of the robot arm. In the experiment with benchmark functions, we transform the search space of existing benchmark functions based on the context vector. Experimental results show that, with nonlinear and noisy transformations, CMA-ES-CWS can obtain the optimal solution corresponding to the given context vector more efficiently than the contextual CMA-ES and WS-CMA-ES. In addition, we confirm that the performance of CMA-ES-CWS is improved by increasing the amount of past optimization results. Further, we evaluate the performance of CMA-ES-CWS in robot control tasks. 2 Preliminaries 2.1 CMA-ES The CMA-ES is a probabilistic model-based black-box continuous optimization method using the multivariate Gaussian distribution \mathcal{N}\left(\boldsymbol{m},\sigma^{2}\boldsymbol{C}\right) which is parameterized by mean vector \boldsymbol{m}^{(t)}\in\mathbb{R}^{N} , covariance matrix \boldsymbol{C}^{(t)}\in\mathbb{R}^{N\times N} , and step-size \sigma^{(t)}\in\mathbb{R}_{>0} . In each iteration, the CMA-ES generates \lambda solutions \boldsymbol{x}^{\langle{1}\rangle},\cdots,\boldsymbol{x}^{\langle{\lambda}\rangle} from the Gaussian distribution as \displaystyle\boldsymbol{z}^{\langle{i}\rangle}\sim\mathcal{N}(\mathbf{0},\mathbf{I})\quad\text{,}\quad\boldsymbol{y}^{\langle{i}\rangle}=\sqrt{\boldsymbol{C}^{(t)}}\boldsymbol{z}^{\langle{i}\rangle}\quad\text{and}\quad\boldsymbol{x}^{\langle{i}\rangle}=\boldsymbol{m}^{(t)}+\sigma^{(t)}\boldsymbol{y}^{\langle{i}\rangle}\kern 5.0pt. (1) Next, the CMA-ES computes the ranking of the solutions on the objective function. We denote the index of the i -th best solution as i\!:\!\lambda . Next, the CMA-ES updates two evolution paths: \boldsymbol{p}_{c}\in\mathbb{R}^{N} and \boldsymbol{p}_{\sigma}\in\mathbb{R}^{N} . These evolution paths are initialized as \boldsymbol{p}_{\sigma}^{(0)}=\boldsymbol{p}_{c}^{(0)}=\mathbf{0} . The CMA-ES computes two weighted sums, namely, \Delta_{\boldsymbol{z}}=\sum_{i=1}^{\mu}w_{i}\boldsymbol{z}^{\langle{i:\lambda}\rangle} and \Delta_{\boldsymbol{y}}=\sum_{i=1}^{\mu}w_{i}\boldsymbol{y}^{\langle{i:\lambda}\rangle} of \mu best solutions, to update the evolution paths as \displaystyle\boldsymbol{p}_{\sigma}^{(t+1)} \displaystyle=(1-c_{\sigma})\boldsymbol{p}_{\sigma}^{(t)}+\sqrt{c_{\sigma}(2-c_{\sigma})\mu_{\mathrm{eff}}}\cdot\Delta_{\boldsymbol{z}} (2) \displaystyle\boldsymbol{p}_{c}^{(t+1)} \displaystyle=(1-c_{c})\boldsymbol{p}_{c}^{(t)}+h_{\sigma}^{(t+1)}\sqrt{c_{c}(2-c_{c})\mu_{\mathrm{eff}}}\cdot\Delta_{\boldsymbol{y}}\kern 5.0pt, (3) where c_{\sigma},c_{c}\in(0,1] are accumulation factors, and \mu_{\mathrm{eff}}=(\sum_{i=1}^{\mu}w_{i}^{2})^{-1} is the variance effective selection mass. The Heaviside function h_{\sigma}\in\{0,1\} is set to h_{\sigma}=1 when it satisfies \displaystyle\frac{\|\boldsymbol{p}_{\sigma}^{(t+1)}\|}{\sqrt{1-(1-c_{\sigma})^{2(t+1)}}}<\left(1.4+\frac{2}{N+1}\right)\chi_{N}\kern 5.0pt, (4) where \chi_{N}=\sqrt{N}\left(1-\frac{1}{4N}+\frac{1}{21N^{2}}\right) denotes the approximated value of \mathbb{E}[\|\mathcal{N}(\mathbf{0},\mathbf{I})\|] . Finally, the CMA-ES updates the distribution parameters. The update rule of the mean vector is given as \displaystyle\boldsymbol{m}^{(t+1)} \displaystyle=\boldsymbol{m}^{(t)}+c_{m}\sigma^{(t)}\Delta_{\boldsymbol{y}}\kern 5.0pt, (5) where the learning rate c_{m}\in(0,1] is usually set as c_{m}=1 . The update of the covariance matrix consists of the rank- \mu update and the rank-one update. The rank- \mu update uses the weighted sum of the \mu best solutions, whereas the rank-one update uses the evolution path \boldsymbol{p}_{c} . The update rule of the covariance matrix is given by \boldsymbol{C}^{(t+1)}=\left(1+(1-h_{\sigma}^{(t+1)})c_{1}c_{c}(2-c_{c})\right)\boldsymbol{C}^{(t)}\\ +\underbrace{c_{\mu}\sum_{i=1}^{\mu}w_{i}\left(\boldsymbol{y}^{\langle{i:\lambda}\rangle}\left(\boldsymbol{y}^{\langle{i:\lambda}\rangle}\right)^{\mathrm{T}}-\boldsymbol{C}^{(t)}\right)}_{\text{rank-$\mu$ update}}+\underbrace{c_{1}\left(\boldsymbol{p}_{c}^{(t+1)}\left(\boldsymbol{p}_{c}^{(t+1)}\right)^{\mathrm{T}}-\boldsymbol{C}^{(t)}\right)}_{\text{rank-one update}}\kern 5.0pt, (6) where c_{\mu},c_{1}\in(0,1] denote the learning rates. The update rule of the step-size is given as \displaystyle\sigma^{(t+1)} \displaystyle=\sigma^{(t)}\exp\left(\frac{c_{\sigma}}{d_{\sigma}}\left(\frac{\|\boldsymbol{p}_{\sigma}^{(t+1)}\|}{\chi_{N}}-1\right)\right)\kern 5.0pt, (7) where d_{\sigma}\in\mathbb{R}_{>0} denotes the damping factor. The CMA-ES has well-tuned recommended settings for each hyperparameter, as shown in [ 10 , 11 ] . The initial distribution parameters \boldsymbol{m}^{(0)},\boldsymbol{C}^{(0)}, and \sigma^{(0)} significantly influence the optimization performance. However, because the precise general-purpose method to determine these important hyperparameters does not exist, they are manually set (or sometimes manually tuned) for the target problem. 2.2 Multi-output Gaussian Process Regression The GPR is a non-parametric regression method, which expresses the prediction using the posterior distribution of the function following a Gaussian process. The Gaussian process is a function g of the distribution whose evaluation values at arbitrary n points \mathcal{D}_{n}=\{\boldsymbol{x}_{i}\}_{i=1}^{n} follow the multivariate Gaussian distribution \mathcal{N}(\mathbf{0},\boldsymbol{K}) . The