Simulating Meaning, Nevermore! Introducing ICR: A Semiotic-Hermeneutic Metric for Evaluating Meaning in LLM Text Summaries

Natalie Perez, Sreyoshi Bhaduri, Aman Chadha · 2026

Read Simulating Meaning, Nevermore! Introducing ICR: A Semiotic-Hermeneutic Metric for Evaluating Meaning in LLM Text Summaries by Natalie Perez, Sreyoshi Bhaduri, Aman Chadha (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-04413.txt.

Description

Perez and colleagues judge meaning in language model summaries with ICR. Meaning is relational and emergent from sign systems, not fixed word to concept maps. The paper reviews signs in static and contextual embeddings and gaps between statistics and human reading. ICR uses inductive content analysis and reflexive thematic analysis to score semantic accuracy beyond word overlap. Across five sets, models match humans on word similarity but lag on semantic accuracy, above all for grounded meanings.

Catalog record

CREATORNatalie Perez, Sreyoshi Bhaduri, Aman Chadha
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE74476
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.04413

Browse the full library · About the Tower of Babel