Enhancing finite-difference-based derivative-free optimization methods with machine learning

Timothé Taminiau, Estelle Massart, Geovani Nunes Grapiglia · 2025

Read Enhancing finite-difference-based derivative-free optimization methods with machine learning by Timothé Taminiau, Estelle Massart, Geovani Nunes Grapiglia (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-07435.txt.

Description

Timothé Taminiau, Estelle Massart, and Geovani Nunes Grapiglia let an optimizer learn from its own scratch work, in a 2025 paper. Derivative-free optimization tunes systems by trying values, never seeing a gradient. Finite-difference versions fake the gradient from nearby tries. Every run generates a rich trail: points tried, values found, approximate slopes, step sizes that worked. Their auxiliary procedure trains a surrogate model on that trail.

The method then descends the surrogate with a guarded line search until the surrogate stops predicting real improvement, falls back to the base algorithm, and retrains on the fresh data. Bolted onto a recent derivative-free method, the loop improved performance significantly in numerical tests, especially when the approximate gradients also trained the surrogate. The authors offer the wrapper as a cheap upgrade for a whole family of optimizers.

Catalog record

CREATORTimothé Taminiau, Estelle Massart, Geovani Nunes Grapiglia
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-07435

Browse the full library · About the Tower of Babel