Capturing Human Cognitive Styles with Language: Towards an Experimental Evaluation Paradigm
Vasudha Varadarajan, Syeda Mahwish, Xiaoran Liu, Julia Buffolino, Christian C. Luhmann, Ryan L. Boyd, H. Andrew Schwartz · 2025
Read Capturing Human Cognitive Styles with Language: Towards an Experimental Evaluation Paradigm by Vasudha Varadarajan, Syeda Mahwish, Xiaoran Liu, Julia Buffolino, Christian C. Luhmann, Ryan L. Boyd, H. Andrew Schwartz (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-13326.txt.
Description
Vasudha Varadarajan and six co-authors test whether the way people talk betrays how they decide. Their 2025 paper complains that language models of the mind are usually graded against labels made without ever observing the person's actual behavior. Their fix is an experiment: measure the behavior first, in the lab, and grade the language against that.
Participants described a recent decision in their own words, then played a classical decision exercise that revealed their style, how their preferences shifted as the task unfolded. Features of their writing, chosen to reflect cognitive style, predicted the measured decision style with solid accuracy, around 0.8 on the standard curve measure. The authors present the paradigm as a template: any claim that language reveals a mental trait should be checkable against a behavioral experiment, not just against annotators' guesses.
Catalog record
| CREATOR | Vasudha Varadarajan, Syeda Mahwish, Xiaoran Liu, Julia Buffolino, Christian C. Luhmann, Ryan L. Boyd, H. Andrew Schwartz |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 44 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-13326