ThoughtTrace : Understanding User Thoughts in Real-World LLM Interactions
Chuanyang Jin 1 , Binze Li 1 , Haopeng Xie 1 , Cathy Mengying Fang 2 , Tianjian Li 1 , Shayne Longpre 2 , Hongxiang Gu 3 , Maximillian Chen 3 , Tianmin Shu 1 · 2026
Read ThoughtTrace : Understanding User Thoughts in Real-World LLM Interactions by Chuanyang Jin 1 , Binze Li 1 , Haopeng Xie 1 , Cathy Mengying Fang 2 , Tianjian Li 1 , Shayne Longpre 2 , Hongxiang Gu 3 , Maximillian Chen 3 , Tianmin Shu 1 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-20087.txt.
Description
Chuanyang Jin and eight co-workers introduce ThoughtTrace, the first large dataset pairing real multi-turn human-AI chats with users' own reported thoughts. It covers 1,058 users, 2,155 conversations and 10,174 thought annotations across 20 language models. Analysis finds thoughts differ semantically from messages, resist inference by frontier models from context alone, vary widely, and track conversation stages. Thoughts also improve user-behaviour prediction as inference-time context, and thought-guided rewrites give fine-grained alignment signals for personalised assistants.
Catalog record
| CREATOR | Chuanyang Jin 1 , Binze Li 1 , Haopeng Xie 1 , Cathy Mengying Fang 2 , Tianjian Li 1 , Shayne Longpre 2 , Hongxiang Gu 3 , Maximillian Chen 3 , Tianmin Shu 1 |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 135781 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.20087