Verifying the Verifiers: Unveiling Pitfalls and Potentials in Fact Verifiers
Wooseok Seo Seungju Han Jaehun Jung, Benjamin Newman Seungwon Lim Seungbeen Lee Ximing Lu, Yejin Choi Youngjae Yu, [1ex] Yonsei University · 2025
Read Verifying the Verifiers: Unveiling Pitfalls and Potentials in Fact Verifiers by Wooseok Seo Seungju Han Jaehun Jung, Benjamin Newman Seungwon Lim Seungbeen Lee Ximing Lu, Yejin Choi Youngjae Yu, [1ex] Yonsei University (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-13342.txt.
Description
Seo, Yu and their co-authors evaluate fact verification systems themselves, in work from 2025. Checking facts is vital for reliable language model use, yet the checkers are rarely checked with the same care.
The team tested many large models and one specialist verifier on examples from a wide set of fact-checking benchmarks. About a sixth of the data proved ambiguous or wrongly labeled, enough to shift model rankings. Big models with a few examples did surprisingly well and should be standard baselines, they argue. Small tuned verifiers still lag on hard reasoning cases, but synthetic multi-step training data helped them.
Catalog record
| CREATOR | Wooseok Seo Seungju Han Jaehun Jung, Benjamin Newman Seungwon Lim Seungbeen Lee Ximing Lu, Yejin Choi Youngjae Yu, [1ex] Yonsei University |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 90 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-13342