Dissecting Atomic Facts: Visual Analytics for Improving Fact Annotations in Language Model Evaluation
Manuel Schmidt University of Konstanz, Daniel A. Keim University of Konstanz, Frederik L. Dennig University of Konstanz · 2025
Read Dissecting Atomic Facts: Visual Analytics for Improving Fact Annotations in Language Model Evaluation by Manuel Schmidt University of Konstanz, Daniel A. Keim University of Konstanz, Frederik L. Dennig University of Konstanz (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-01460.txt.
Description
Checking whether a language model's output is factual starts by splitting its text into small atomic facts, but annotators often disagree on how to split. Manuel Schmidt, Daniel A. Keim, and Frederik L. Dennig present a visual analytics concept for exposing those disagreements in fact annotation. Vague definitions of atomicity, they argue, leave too much ambiguity in fact extraction.
Their approach visualizes semantic alignment, granularity, and referential links between extracted facts, so patterns of inconsistency become visible at a glance. Guided revision loops then help annotators converge on shared choices. The goal is a steadier base for factuality benchmarks, which in turn makes model evaluation more trustworthy. Schmidt and the co-authors treat annotation quality as a design problem that visual tools can help solve, rather than a matter of stricter instructions alone.
Catalog record
| CREATOR | Manuel Schmidt University of Konstanz, Daniel A. Keim University of Konstanz, Frederik L. Dennig University of Konstanz |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 10 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-01460