LLM-Metrics: Measuring Research Impact Through Large Language Model Memory

Si Shen, Wenhua Zhao, Danhao Zhu · 2026

Read LLM-Metrics: Measuring Research Impact Through Large Language Model Memory by Si Shen, Wenhua Zhao, Danhao Zhu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-22176.txt.

Description

Si Shen and co-authors propose LLM-Metrics, a research-impact metric drawn from large language model memory. Citation counts suffer lag, disciplinary bias, and Matthew effects, so the authors test whether high-impact papers leave stronger parametric memory through greater exposure. Four probe types cover title, author, method, and venue recognition across 549 papers from 2023-2024 and 17 models. Fifteen models give positive predictions, with overall Spearman correlation 0.1495 against citations. Author-recognition probes prove strongest, and a 3B model beats most larger ones, supporting a selective-memory account.

Catalog record

CREATORSi Shen, Wenhua Zhao, Danhao Zhu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE63294
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.22176

Browse the full library · About the Tower of Babel