Enhancing finite-difference-based derivative-free optimization methods with machine learning
Timothé Taminiau, Estelle Massart, Geovani Nunes Grapiglia · 2025
Read Enhancing finite-difference-based derivative-free optimization methods with machine learning by Timothé Taminiau, Estelle Massart, Geovani Nunes Grapiglia (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-07435.txt.
Description
Timothé Taminiau, Estelle Massart, and Geovani Nunes Grapiglia let an optimizer learn from its own scratch work, in a 2025 paper. Derivative-free optimization tunes systems by trying values, never seeing a gradient. Finite-difference versions fake the gradient from nearby tries. Every run generates a rich trail: points tried, values found, approximate slopes, step sizes that worked. Their auxiliary procedure trains a surrogate model on that trail.
The method then descends the surrogate with a guarded line search until the surrogate stops predicting real improvement, falls back to the base algorithm, and retrains on the fresh data. Bolted onto a recent derivative-free method, the loop improved performance significantly in numerical tests, especially when the approximate gradients also trained the surrogate. The authors offer the wrapper as a cheap upgrade for a whole family of optimizers.
Catalog record
| CREATOR | Timothé Taminiau, Estelle Massart, Geovani Nunes Grapiglia |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-07435