Dissecting Atomic Facts: Visual Analytics for Improving Fact Annotations in Language Model Evaluation

Manuel Schmidt University of Konstanz, Daniel A. Keim University of Konstanz, Frederik L. Dennig University of Konstanz · 2025

Read Dissecting Atomic Facts: Visual Analytics for Improving Fact Annotations in Language Model Evaluation by Manuel Schmidt University of Konstanz, Daniel A. Keim University of Konstanz, Frederik L. Dennig University of Konstanz (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-01460.txt.

Description

Checking whether a language model's output is factual starts by splitting its text into small atomic facts, but annotators often disagree on how to split. Manuel Schmidt, Daniel A. Keim, and Frederik L. Dennig present a visual analytics concept for exposing those disagreements in fact annotation. Vague definitions of atomicity, they argue, leave too much ambiguity in fact extraction.

Their approach visualizes semantic alignment, granularity, and referential links between extracted facts, so patterns of inconsistency become visible at a glance. Guided revision loops then help annotators converge on shared choices. The goal is a steadier base for factuality benchmarks, which in turn makes model evaluation more trustworthy. Schmidt and the co-authors treat annotation quality as a design problem that visual tools can help solve, rather than a matter of stricter instructions alone.

Catalog record

CREATORManuel Schmidt University of Konstanz, Daniel A. Keim University of Konstanz, Frederik L. Dennig University of Konstanz
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE10 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-01460

Browse the full library · About the Tower of Babel