RealFin: How Well Do LLMs Reason About Finance When Users Leave Things Unsaid?

Yuyang Dai, Yan Lin, Zhuohan Xie, Yuxia Wang · 2026

Read RealFin: How Well Do LLMs Reason About Finance When Users Leave Things Unsaid? by Yuyang Dai, Yan Lin, Zhuohan Xie, Yuxia Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-07096.txt.

Description

Yuyang Dai and three co-authors test how well language models reason about finance when users leave things unsaid. Sound financial reasoning needs more than answering well; it needs knowing when no answer can be backed up. In real practice, problems often rest on implicit assumptions taken for granted, not stated outright. They can look answerable though they lack what a firm answer needs. The team's bilingual benchmark, REALFIN, removes essential premises from exam-style questions in a systematic way, while keeping the questions linguistically plausible.

Models face three formulations: answering, spotting missing information, and rejecting unjustified options. Scores fall consistently when key conditions are absent. Models built for general use tend to commit too soon, and to guess. Most finance-specialized models fail to clearly flag missing premises. The gap shows current tests miss a key skill, and reliable financial models must know when to leave a question unanswered.

Catalog record

CREATORYuyang Dai, Yan Lin, Zhuohan Xie, Yuxia Wang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE93 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-07096

Browse the full library · About the Tower of Babel