GPT Meets Graphs and KAN Splines: Testing Novel Frameworks on Multitask Fine-Tuned GPT-2 with LoRA Stanford CS224N Default Project
Marc Bernardino, Gabriel Bo, Justin Gu · 2025
Read GPT Meets Graphs and KAN Splines: Testing Novel Frameworks on Multitask Fine-Tuned GPT-2 with LoRA Stanford CS224N Default Project by Marc Bernardino, Gabriel Bo, Justin Gu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-10490.txt.
Description
Marc Bernardino, Gabriel Bo, and Justin Gu test new modules inside a GPT-2 model in a 2025 paper. They ask whether Kolmogorov-Arnold Networks and graph based parts can help a pretrained model learn several tasks and stay easier to read. The base model is first tuned with LoRA, a light adaptation method, plus careful settings.
That tuned base does very well on feeling detection, paraphrase detection, and sonnet writing. Two new variants, Graph LoRA and a hybrid KAN version, are then built and tested the same way. Neither beats the plain tuned model, the authors report. Light adaptation remains the best strategy here.
Catalog record
| CREATOR | Marc Bernardino, Gabriel Bo, Justin Gu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 23 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-10490