Aligned Textual Scoring Rules
Yuxuan Lu, Yifan Wu, Jason Hartline, Michael J. Curry · 2025
Read Aligned Textual Scoring Rules by Yuxuan Lu, Yifan Wu, Jason Hartline, Michael J. Curry (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-06221.txt.
Description
Yuxuan Lu, Yifan Wu, Jason Hartline, and Michael Curry work on honest forecasting when the forecast is written in words. Their paper came out in two thousand twenty-five. Scoring rules reward numeric forecasts so that telling the truth pays best. Newer work maps text forecasts onto those numeric rules. Yet a rule can be honest and still clash with what people judge as good text.
The team designs an Aligned Scoring rule for text. It starts from a proper scoring rule, then bends it toward a reference score such as human judgment. The bend is chosen to cut the squared gap between the two. Tests show better fit with human taste than earlier methods. Honesty is kept while the fit improves. The work matters as language models report beliefs in prose.
Catalog record
| CREATOR | Yuxuan Lu, Yifan Wu, Jason Hartline, Michael J. Curry |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 53 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-06221