Simulating Meaning, Nevermore! Introducing ICR: A Semiotic-Hermeneutic Metric for Evaluating Meaning in LLM Text Summaries
Natalie Perez, Sreyoshi Bhaduri, Aman Chadha · 2026
Read Simulating Meaning, Nevermore! Introducing ICR: A Semiotic-Hermeneutic Metric for Evaluating Meaning in LLM Text Summaries by Natalie Perez, Sreyoshi Bhaduri, Aman Chadha (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-04413.txt.
Description
Perez and colleagues judge meaning in language model summaries with ICR. Meaning is relational and emergent from sign systems, not fixed word to concept maps. The paper reviews signs in static and contextual embeddings and gaps between statistics and human reading. ICR uses inductive content analysis and reflexive thematic analysis to score semantic accuracy beyond word overlap. Across five sets, models match humans on word similarity but lag on semantic accuracy, above all for grounded meanings.
Catalog record
| CREATOR | Natalie Perez, Sreyoshi Bhaduri, Aman Chadha |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 74476 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.04413