Be aware of overfitting by hyperparameter optimization!
Igor V. Tetko, Ruud van Deursen, Guillaume Godin · 2024
Read Be aware of overfitting by hyperparameter optimization! by Igor V. Tetko, Ruud van Deursen, Guillaume Godin (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/pmc-11629497.txt.
Description
Tuning a machine learning model's settings can quietly cheat: search hard enough and the settings fit the test data's noise. Igor Tetko, Ruud van Deursen, and Guillaume Godin demonstrated in 2024 this trap in drug solubility prediction.
Re-examining seven datasets, they showed heavy tuning often failed to beat preset settings, while costing about ten thousand times more compute. Their own entrant, a transformer model reading molecule strings as language, beat graph-based rivals in twenty-six of twenty-eight comparisons, in a fraction of the time. Their moral: preset settings travel safer, and fair comparisons demand identical scoring measures.
Catalog record
| CREATOR | Igor V. Tetko, Ruud van Deursen, Guillaume Godin |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55 KB |
| VERSION | 1.0 |
| LICENSE | CC BY 4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |