Mind the Gap! Choice Independence in Using Multilingual LLMs for Persuasive Co-Writing Tasks in Different Languages
Shreyan Biswas, Erlei, Alexander, Ujwal Gadiraju · 2025
Read Mind the Gap! Choice Independence in Using Multilingual LLMs for Persuasive Co-Writing Tasks in Different Languages by Shreyan Biswas, Erlei, Alexander, Ujwal Gadiraju (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-09532.txt.
Introduction
Biswas, Erlei, and Gadiraju test if tool flaws in one tongue sway use in a next. New write aides lean on tools that speak more than one tongue. Such tools vary in grade by tongue, by wide past proof. Users of more than one tongue thus face split tool grades. Men oft spread a tool's flaws cross lone tasks in past work. That spread breaks the choice-free rule in act terms.
Their task is a gift ad write job in two tongues. They ask if aide grade in tongue two bends use in tongue one. They also weigh how that bend shapes the ad's pull. Faith in tool use joins the weigh, as do gift picks. Finds show aides' users do break choice-free acts in the task. Grade in one tongue bent trust and use in the next.
Catalog record
| CREATOR | Shreyan Biswas, Erlei, Alexander, Ujwal Gadiraju |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 91 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-09532
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Mind the Gap! Choice Independence in Using Multilingual LLMs for Persuasive Co-Writing Tasks in Different Languages Shreyan Biswas, Erlei, Alexander, Ujwal Gadiraju Abstract . Recent advances in generative AI have precipitated a proliferation of novel writing assistants. These systems typically rely on multilingual large language models (LLMs), providing globalized workers the ability to revise or create diverse forms of content in different languages. However, there is substantial evidence indicating that the performance of multilingual LLMs varies between languages. Users who employ writing assistance for multiple languages are therefore susceptible to disparate output quality. Importantly, recent research has shown that people tend to generalize algorithmic errors across independent tasks, violating the behavioral axiom of choice independence . In this paper, we analyze whether user utilization of novel writing assistants in a charity advertisement writing task is affected by the AI’s performance in a second language. Furthermore, we quantify the extent to which these patterns translate into the persuasiveness of generated charity advertisements, as well as the role of peoples’ beliefs about LLM utilization in their donation choices. Our results provide evidence that writers who engage with an LLM-based writing assistant violate choice independence, as prior exposure to a Spanish LLM reduces subsequent utilization of an English LLM. While these patterns do not affect the aggregate persuasiveness of the generated advertisements, people’s beliefs about the source of an advertisement (human versus AI) do. In particular, Spanish-speaking female participants who believed that they read an AI-generated advertisement strongly adjusted their donation behavior downwards. Furthermore, people are generally not able to adequately differentiate between human-generated and LLM-generated ads. Our work has important implications for the design, development, integration, and adoption of multilingual LLMs as assistive agents—particularly in writing tasks. DOI: XXXXXXX.XXXXXXX Conference: Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems; April 26– May 1, 2025; Yokohama, Japan ISBN: 978-1-4503-XXXX-X/18/06 Conference: CHI Conference on Human Factors in Computing Systems; April 26-May 1, 2025; Yokohama, Japan CHI Conference on Human Factors in Computing Systems (CHI ’25), April 26-May 1, 2025, Yokohama, Japan DOI: 10.1145/3706598.3713201 ISBN: 979-8-4007-1394-1/25/04 CCS: Human-centered computing Empirical studies in HCI CCS: Human-centered computing User studies CCS: Applied computing Economics CCS: Applied computing Psychology CCS: Human-centered computing Empirical studies in collaborative and social computing † † cc-license: by 1. Introduction The advent of large language models (LLMs) has transformed human-AI co-writing ( Reza et al., 2023 ) . Modern writing assistants such as Microsoft 365 Copilot, Grammarly, or Jasper leverage multilingual LLMs to support a global user base in drafting, editing, and rewriting content ( Noy and Zhang, 2023 ; Herbold et al., 2023b ) . However, despite their widespread use, LLMs still differ substantially between languages ( Ahuja et al., 2023 ; Zhang et al., 2024a ; Huang et al., 2023 ; Bang et al., 2023 ; Joshi et al., 2020 ; Jiao et al., 2023 ) . While English is the default language and therefore exhibits robust performance, lower resource languages often display deficiencies in fluency, coherence, sensitivity to jailbreaks, and contextual appropriateness ( Conde et al., 2024 ; Yong et al., 2023 ; Hasan et al., 2024 ; Lawal et al., 2024 ) . These performance disparities in multilingual LLMs can have significant behavioral implications . Users interacting with LLMs in multilingual settings may not only be influenced by technical limitations but also by their perceptions and experiences from being exposed to heterogeneous output quality. Consider a bilingual journalist writing news articles in both English and Spanish. When using an LLM assistant, they notice it struggles with Spanish idioms and cultural references, requiring substantial editing. A rational writer would evaluate each language independently, maximizing the LLM’s benefits in English while being more selective in Spanish. Yet, recent evidence has shown that humans use prior experience with LLMs to predict future performance ( Rambachan and Others, 2024 ) and, in particular, often violate the independence axiom of rational choice theory by generalizing AI errors to objectively unrelated tasks ( Erlei et al., 2024 ) . In the example above, this may cause the journalist to avoid the LLM for English articles due to a faulty generalization of their Spanish experience. Similar arguments apply to all regions or domains where linguistic diversity is high, such as customer support, technical documentation, or professional writing. Understanding how users react to heterogeneity in these scenarios is important, particularly given the current trend where LLM assistants are incrementally rolled out in multiple languages to meet global demands ( Sumner, 2024 ; Meta, 2024 ; Venkatachary, 2024 ) . Importantly, there are good reasons to doubt that these behavioral patterns would automatically resolve solely through experience and learning-by-doing. First, human-AI interaction research highlights the critical role of first impressions and initial exposures for user behaviour and trust ( Schwartz et al., 2023 ; Nourani et al., 2020 ; Glikson and Woolley, 2020 ; Tolmeijer et al., 2021 ) . Even with repeated zero-stakes interactions, individuals frequently fail to optimally calibrate their reliance on AI systems after observing initial errors, exhibiting persistent downward bias in utilization patterns. In real-world settings, learning about LLM capabilities is costly, which can further deter users from updating their beliefs. Second, it is unclear whether choice independence violations stem from biased information updating or a biased choice rule. While the latter may improve over time, biased information updating inhibits learning and hence perpetuates inefficient under-utilization. In our study, 1 1 1 The codebase and datasets are openly available: https://sites.google.com/view/cillm-mind-the-gap we examine how exposure to two different languages affects people’s interactions with an LLM-based writing assistant in the context of persuasive writing. We focus on English and Spanish, two high-resource languages that allow us to commission multiple advertisements from writers and subsequently test them in a charitable giving task. Participants in the first experiment wrote two advertisements for the charity World Wildlife Fund (WWF), one in English and one in Spanish, with the explicitly incentivized assignment to make it as persuasive as possible. By varying the order of the language, we measure how exposure to the lower (higher) resource language Spanish (English) affects subsequent interaction with our LLM-based co-writing tool in the higher (lower) resource language English (Spanish). For comparison, we also ask native English and Spanish speakers to write advertisements without any AI assistance. The second experiment then evaluates the generated ads via a charitable giving task in which participants split an endowment of £1.5 between themselves and the WWF after reading a treatment-dependent advertisement. In addition to the ads from the first experiment, we also evaluate advertisements that are purely derived from LLMs, as well as the WWF mission statement. Finally, we elicit donors’ beliefs about the origin of the advertisement (human versus AI). Through this empirical setup, we address the following four research questions: RQ 1: How does LLM performance in one language affect user utilization in a second language for persuasive co-writing? RQ 2: How do varying levels of LLM utilization in co-writing tasks across languages influence the persuasiveness of generated advertisements? RQ 3: How does altruism persuasiveness differ between human writers, human-LLM teams, and LLMs? RQ 4: How do donor beliefs about the source of an advertisement affect altruistic behaviour? Our results confirm that writers violate