GPT Meets Graphs and KAN Splines: Testing Novel Frameworks on Multitask Fine-Tuned GPT-2 with LoRA Stanford CS224N Default Project

Marc Bernardino, Gabriel Bo, Justin Gu · 2025

Read GPT Meets Graphs and KAN Splines: Testing Novel Frameworks on Multitask Fine-Tuned GPT-2 with LoRA Stanford CS224N Default Project by Marc Bernardino, Gabriel Bo, Justin Gu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-10490.txt.

Description

Marc Bernardino, Gabriel Bo, and Justin Gu test new modules inside a GPT-2 model in a 2025 paper. They ask whether Kolmogorov-Arnold Networks and graph based parts can help a pretrained model learn several tasks and stay easier to read. The base model is first tuned with LoRA, a light adaptation method, plus careful settings.

That tuned base does very well on feeling detection, paraphrase detection, and sonnet writing. Two new variants, Graph LoRA and a hybrid KAN version, are then built and tested the same way. Neither beats the plain tuned model, the authors report. Light adaptation remains the best strategy here.

Catalog record

CREATORMarc Bernardino, Gabriel Bo, Justin Gu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE23 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-10490

Browse the full library · About the Tower of Babel