Exploring Coding Spot : Understanding Parametric Contributions to LLM Coding Performance

Dongjun Kim, Minhyuk Kim, YongChan Chun, Chanjun Park, Heuiseok LimKorea University {junkim100, mhkim0929, cyc9805, bcj1210, limhseok}@korea.ac.kr · 2024

Read Exploring Coding Spot : Understanding Parametric Contributions to LLM Coding Performance by Dongjun Kim, Minhyuk Kim, YongChan Chun, Chanjun Park, Heuiseok LimKorea University {junkim100, mhkim0929, cyc9805, bcj1210, limhseok}@korea.ac.kr (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-07113.txt.

Description

Dongjun Kim and co-authors published a research paper in 2024 on a small set of parameters inside language models that seems to carry their coding skill. Brains park skills in set regions; the team asked whether models do the same for programming. They call the suspected region the Coding Spot.

They located the region and then altered just those parameters. Coding test scores swung sharply, while skills outside coding stayed mostly intact. The split held across the tasks they tried. If skills really live in pockets like this, tuning a model for one trade may not require touching the rest. The paper opens a path to surgical, skill-by-skill model edits.

Catalog record

CREATORDongjun Kim, Minhyuk Kim, YongChan Chun, Chanjun Park, Heuiseok LimKorea University {junkim100, mhkim0929, cyc9805, bcj1210, limhseok}@korea.ac.kr
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE28 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-07113

Browse the full library · About the Tower of Babel