ThoughtTrace : Understanding User Thoughts in Real-World LLM Interactions

Chuanyang Jin 1 , Binze Li 1 , Haopeng Xie 1 , Cathy Mengying Fang 2 , Tianjian Li 1 , Shayne Longpre 2 , Hongxiang Gu 3 , Maximillian Chen 3 , Tianmin Shu 1 · 2026

Read ThoughtTrace : Understanding User Thoughts in Real-World LLM Interactions by Chuanyang Jin 1 , Binze Li 1 , Haopeng Xie 1 , Cathy Mengying Fang 2 , Tianjian Li 1 , Shayne Longpre 2 , Hongxiang Gu 3 , Maximillian Chen 3 , Tianmin Shu 1 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-20087.txt.

Description

Chuanyang Jin and eight co-workers introduce ThoughtTrace, the first large dataset pairing real multi-turn human-AI chats with users' own reported thoughts. It covers 1,058 users, 2,155 conversations and 10,174 thought annotations across 20 language models. Analysis finds thoughts differ semantically from messages, resist inference by frontier models from context alone, vary widely, and track conversation stages. Thoughts also improve user-behaviour prediction as inference-time context, and thought-guided rewrites give fine-grained alignment signals for personalised assistants.

Catalog record

CREATORChuanyang Jin 1 , Binze Li 1 , Haopeng Xie 1 , Cathy Mengying Fang 2 , Tianjian Li 1 , Shayne Longpre 2 , Hongxiang Gu 3 , Maximillian Chen 3 , Tianmin Shu 1
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE135781
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.20087

Browse the full library · About the Tower of Babel