ShareChat : A Dataset of Chatbot Conversations in the Wild

Yueru Yan, Tuc Nguyen, Bo Su, Melissa Lieffers, Thai Le · 2025

Read ShareChat : A Dataset of Chatbot Conversations in the Wild by Yueru Yan, Tuc Nguyen, Bo Su, Melissa Lieffers, Thai Le (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-17843.txt.

Introduction

Yueru Yan and four coauthors built ShareChat, a large collection of real chatbot conversations gathered from links that users shared in public. The corpus holds 142,808 conversations and over 660,000 turns from five platforms: ChatGPT, Claude, Gemini, Perplexity, and Grok. Unlike older datasets, it keeps each platform's own features, such as citations and code.

Three case studies show what the data can reveal. One compares how fully each platform meets user intent, another compares how search based systems cite sources, and a third tracks how reply speed changed over time. The set covers 95 languages from April 2023 to October 2025 and is open to other researchers.

Catalog record

CREATORYueru Yan, Tuc Nguyen, Bo Su, Melissa Lieffers, Thai Le
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE64 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-17843

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ShareChat : A Dataset of Chatbot Conversations in the Wild Yueru Yan, Tuc Nguyen, Bo Su, Melissa Lieffers, Thai Le Abstract While Large Language Models (LLMs) have evolved into distinct platforms with unique interface designs and capabilities, existing public datasets treat models as generic text generators, stripping away the interface context that actively shapes user interaction. To address this limitation, we present ShareChat , a large-scale, cross-platform corpus comprising 142,808 conversations and over 660,000 turns collected from publicly shared URLs across five major platforms: ChatGPT, Claude, Gemini, Perplexity, and Grok. ShareChat distinguishes itself by preserving native platform affordances often lost in standard logs, including reasoning traces, source links, and code artifacts, while spanning 101 languages over the period from April 2023 to October 2025. Furthermore, ShareChat offers substantially longer context windows and greater interaction depth than prior datasets. We demonstrate the dataset’s multifaceted utility through three demonstrative use cases: (1) analyzing conversation completeness to measure user intent satisfaction; (2) evaluating source citation behaviors in content generation; and (3) conducting temporal analysis to track evolving usage patterns. This work provides the community with a vital and timely resource for understanding authentic user-LLM chatbot interactions in the wild 1 Introduction Conversational large language model (LLM)-based chatbot services have evolved rapidly in the past three years. The first widely adopted general purpose LLM chatbot, ChatGPT, was launched in November 2022 and has reached more than 700 million weekly active users by mid 2025 OpenAI Research 2025 . Following this success, many companies quickly released their own chatbot services, which also attracted substantial public attention. For example, Anthropic introduced the Claude families in 2023 Anthropic 2023 ; Anthropic 2024 ; Anthropic 2025 , Google deployed the Gemini family of models Anil et al. 2023 , xAI launched Grok as a chatbot integrated with the social media platform X xAI 2023 , Perplexity emerged as an answer engine that combines conversational interaction with web search Perplexity AI 2024 , and DeepSeek released its R1 reasoning model and corresponding consumer chatbot app in January 2025 DeepSeek AI 2025 . Although all of these services are built on text-based LLMs, they differ in interface design, supported features, and safety policies, which in turn shape how users interact with them. For example, Grok can surface live posts from X when providing answers, Claude models are optimized and evaluated for coding, math, and analysis tasks Anthropic 2023 ; Anthropic 2024 , and Perplexity consistently presents responses with explicit source citations Perplexity AI 2024 . These platform level design choices are not static. Commercial systems typically undergo continuous reinforcement learning and fine tuning on conversation logs and human feedback Ouyang et al. 2022 ; Bai et al. 2022 ; Chen et al. 2024 , which tends to reinforce platform specific strengths and norms over time. As a result, it is important to understand interaction dynamics and user experience across platforms, instead of treating LLM capabilities as a property of models in isolation. However, most widely used LLM benchmarks focus on curated single turn tasks, such as multiple choice knowledge tests or coding and math word problems Hendrycks et al. 2021 ; Cobbe et al. 2021 . In practice, the use of conversational LLMs is much more diverse and complex: users ask follow up questions, change goals, or co construct content over many turns. Recent work has shown that even state of the art models often get “lost” in multi turn conversations, with reliability degrading as instructions are refined across turns Laban et al. 2025a . Studying how people actually interact with LLMs in the wild therefore requires large scale, real conversation datasets, rather than only synthetic prompts or short, benchmark style dialogues. Several real world chatbot LLM conversation datasets have been released in the past two years Zhao et al. 2024 ; Zheng et al. 2024 ; Köpf et al. 2023 . For instance, WildChat Zhao et al. 2024 compiles about one million conversations between users and ChatGPT collected via an access gateway that offered free usage in exchange for consent to share logs. LMSYS-Chat-1M Zheng et al. 2024 contains one million conversations with twenty five state of the art models recorded through a single web interface (the Vicuna demo and Chatbot Arena). OpenAssistant Conversations Köpf et al. 2023 provides human generated assistant style dialogues and detailed preference annotations to support alignment research. These datasets are important resources, but they have several limitations: • Single-Interface Mediation: They are all mediated via a central interface even when multiple backend models are available. Consequently, they do not fully capture how different commercial platforms, with distinct interaction designs and product positioning, influence the content that users share and the way they engage with LLMs. • Text-Only Representation: In addition, existing real conversation datasets primarily focus on plain text message content. Yet real interactions with contemporary LLM systems can include more structure: users may enable a thinking mode that exposes intermediate reasoning traces, and some platforms attach source links or other metadata to model outputs, such as the cited web pages surfaced by Perplexity or embedded X posts shown by Grok. A richer multi platform dataset should preserve these elements when they are available, so that researchers can better understand how users interpret model reasoning, how they rely on external evidence, and how platform affordances shape behavior across turns. • Limited Interaction Depth: Moreover, although some datasets explicitly include multi turn conversations Zhao et al. 2024 , the average depth of a conversation is still relatively short. Given that context length is central to LLM training and performance, longer real user chatbot conversations are valuable both for understanding how user intents and prompts evolve and for investigating when and how models succeed or fail at using extended context Laban et al. 2025a . • Observer Bias: Finally, data collection in these studies typically relies on pre-interaction consent, where users agree to share their data before or during the interaction in exchange for service access. This introduces the Observer Bias (or Hawthorne Effect) McCambridge et al. 2014 , as users who are aware they are being monitored for research purposes may consciously or unconsciously alter their behavior, which often exhibits higher social desirability compared to authentic, unobserved private usage. In contrast, ShareChat relies on post-hoc sharing, where the decision to publish is made only after the natural interaction has concluded, thereby preserving the integrity of the original behavior during the conversation itself. In this study, we address these gaps by constructing a large scale, multi platform corpus of authentic user-LLM conversations. We compile 142,808 conversations with more than 660,293 interaction turns from five widely used platforms: Claude, Perplexity, Grok, ChatGPT, and Gemini. All conversations are collected from URLs that users chose to share publicly on the respective platforms, spanning more than twenty months of AI chat usage (April 2023 to October 2025) and covering 101 distinct languages. The dataset therefore captures authentic usage patterns from a diverse, global user base, rather than simulated prompts or tasks designed by researchers. Each conversation record contains the complete sequence of user and assistant turns, along with platform specific metadata such as, when available, timestamps, model thinking traces, or source links attached by the interface. The data collection is

Read the complete text