Learning More May Not Be Better: Knowledge Transferability in Vision-and-Language Tasks
Tianwei Chen, Noa Garcia, Mayu Otani, Chenhui Chu, Yuta Nakashima, Hajime Nagahara, Yanfeng Li · 2024
Read Learning More May Not Be Better: Knowledge Transferability in Vision-and-Language Tasks by Tianwei Chen, Noa Garcia, Mayu Otani, Chenhui Chu, Yuta Nakashima, Hajime Nagahara, Yanfeng Li (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/pmc-11676753.txt.
Description
Training a vision-and-language model on more tasks can hurt, not help, Chen and co-authors warn in a 2024 paper. The field often assumes that pooling datasets lifts all tasks. The team ran hundreds of cross-tests across twelve tasks in four groups, moving knowledge from each task to the others.
Tasks in the same group often helped each other, but not always. Some transfers lowered scores on the target task. Dataset size and the stage of pre-training also swayed results a great deal. The authors map when extra learning pays and when it backfires. Model builders can use the map to pick training mixes with care instead of pooling blindly.
Catalog record
| CREATOR | Tianwei Chen, Noa Garcia, Mayu Otani, Chenhui Chu, Yuta Nakashima, Hajime Nagahara, Yanfeng Li |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 72 KB |
| VERSION | 1.0 |
| LICENSE | CC BY 4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |