LLM-Metrics: Measuring Research Impact Through Large Language Model Memory
Si Shen, Wenhua Zhao, Danhao Zhu · 2026
Read LLM-Metrics: Measuring Research Impact Through Large Language Model Memory by Si Shen, Wenhua Zhao, Danhao Zhu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-22176.txt.
Description
Si Shen and co-authors propose LLM-Metrics, a research-impact metric drawn from large language model memory. Citation counts suffer lag, disciplinary bias, and Matthew effects, so the authors test whether high-impact papers leave stronger parametric memory through greater exposure. Four probe types cover title, author, method, and venue recognition across 549 papers from 2023-2024 and 17 models. Fifteen models give positive predictions, with overall Spearman correlation 0.1495 against citations. Author-recognition probes prove strongest, and a 3B model beats most larger ones, supporting a selective-memory account.
Catalog record
| CREATOR | Si Shen, Wenhua Zhao, Danhao Zhu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 63294 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.22176