On Splitting Training and Validation Set: A Comparative Study of Cross-Validation, Bootstrap and Systematic Sampling for Estimating the Generalization Performance of Supervised Learning.
Xu Y, Goodacre R. · 2018
Read On Splitting Training and Validation Set: A Comparative Study of Cross-Validation, Bootstrap and Systematic Sampling for Estimating the Generalization Performance of Supervised Learning. by Xu Y, Goodacre R. (2018) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/europmc-pmc6373628.txt.
Description
Xu and Goodacre present a comparative study in the Journal of Analysis and Testing. They test how the split of data into training and test parts changes judged model quality. Good checks matter, since scores on training data alone can hide overfitting and flatter a weak model.
They made nine simulated datasets and built two kinds of classifiers on each split. The splits used cross-validation, bootstrap forms, and fixed sampling plans such as Kennard-Stone and SPXY. On small datasets, judged quality missed true test quality by a wide gap for all methods. The gap shrank as datasets grew and models neared stable behavior. Fixed sampling plans gave the worst estimates. The split sizes also needed a fair balance to judge quality well.
Catalog record
| CREATOR | Xu Y, Goodacre R. |
|---|---|
| YEAR | 2018 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 42 KB |
| LICENSE | CC BY 4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |