The Algorithmic Advantage: How Reinforcement Learning Generates Rich Communication Conference: ; ;

Emilio Calvano, Clemens Possnig, Juha Tolvanen · 2026

Read The Algorithmic Advantage: How Reinforcement Learning Generates Rich Communication Conference: ; ; by Emilio Calvano, Clemens Possnig, Juha Tolvanen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-12035.txt.

Introduction

Cheap talk theory assumes a fully rational advisor, but real advice now often comes from learning systems. Emilio Calvano and two co-authors study strategic communication when the advisor is a reinforcement-learning algorithm adapting messages from payoff feedback while the decision maker best-responds. Their theoretical analysis tracks long-run outcomes under reward-driven adaptation.

With aligned goals, learning, as a rule, produces informative messages even from uninformative starting policies. With clashing preferences, no stable outcome exists; instead, learning generates cycles that sustain highly open talk with payoffs above any static standoff. The algorithmic advisor, the paper argues, can beat textbook benchmarks precisely because learning never settles: cycles carry information that equilibria cannot.

Catalog record

CREATOREmilio Calvano, Clemens Possnig, Juha Tolvanen
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE98 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-12035

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

The Algorithmic Advantage: How Reinforcement Learning Generates Rich Communication Conference: ; ; Emilio Calvano, Clemens Possnig, Juha Tolvanen Abstract . We analyze strategic communication when advice is generated by a reinforcement-learning algorithm rather than by a fully rational sender. Building on the cheap-talk framework of Crawford and Sobel (1982), an advisor adapts its messages based on payoff feedback, while a decision maker best-responds. We provide a theoretical analysis of the long-run communication outcomes induced by such reward-driven adaptation. With aligned preferences, we establish that learning robustly leads to informative communication even from uninformative initial policies. With misaligned preferences, no stable outcome exists; instead, learning generates cycles that sustain highly informative communication and payoffs exceeding those of any static equilibrium. 1. Introduction Suppose an advisor observes a stream of data containing information that is payoff relevant for a decision maker. The advisor employs an algorithm that periodically distills this data stream into a message, transmits it to the decision maker, and observes her subsequent choice. Through this feedback, the algorithm learns to adjust its messaging strategy so as to influence the decision maker’s choices in a direction aligned with the advisor’s objectives. Does the algorithm learn to communicate meaningfully? How much information is ultimately transmitted? And does common knowledge of the strategic intent hinder the learning process? These questions are of substantial economic importance as algorithmic advice increasingly shapes decisions in diverse domains. For example, large digital platforms extract information from big data and use it to inform and/or influence user behavior/engagement all while users may ignore advice due to biases or incentive misalignment. Recent machine and reinforcement learning methods tackle these challenges jointly, directly optimizing communication policies that induce better decisions without requiring explicit interpretation of the informational content of the underlying data. To make this concrete, consider a rental platform like Airbnb that provides hosts with daily price suggestions. The platform’s problem is to periodically turn a wealth of time-varying demand data into actionable, unit-specific price advice that hosts will actually find in their best interest to follow. The canonical economics approach is to estimate demand and then compute and effectively communicate the optimal price. However, doing this reliably at scale is complex and model-intensive. Reinforcement learning (RL) offers a more direct, model-free route by learning a mapping from data to price suggestions without explicitly solving the underlying pricing problem. To understand how reinforcement learning shapes information transmission, we analyze this problem by embedding an RL algorithm within the well-known cheap talk framework of Crawford and Sobel (1982). Specifically, we assume that the sender acts like a so-called tabular Q-learning algorithm, while the receiver behaves like a sequence of short-lived idealized economic agents always best-responding to the sender’s current strategy (or, in one of our extensions, learning faster than the sender). This allows us to talk to applications where human decision makers receive advice from an algorithmic recommendation system learning to best respond to the advice over time. We study both an environment where the receiver’s preferences are perfectly aligned with the sender, as well as the traditional environment where the sender prefers higher actions than the receiver. The key contribution of this paper is to characterize the simple force that drives the evolution of communication under learning which we refer to as reward-driven adaptation . This mechanism reinforces policy changes that generate higher payoffs for the sender and attenuates those that do not, thereby acting as a powerful selection force over communication policies. We show that reward-driven adaptation disciplines the learning process leading to sharp theoretical guarantees and robustly informative communication while never reaching the welfare maximizing level. In the baseline, no-bias environment we provide analytical results showing that any stable long-run outcome of the learning process must transmit a substantial amount of payoff-relevant information about the state: welfare is bounded away from the babbling benchmark by a tight lower bound. Importantly, this guarantee does not hinge on knife-edge mixing, or favorable initial conditions, and it can arise even when the sender is initialized at a completely uninformative policy. Specifically, using a natural measure of payoff-relevant informativeness, we show that any attracting policy attains at least a tight lower-bound welfare level, which in our baseline calibration equals at least 98 percent of the potential surplus that could be generated under full revelation. When the sender prefers higher actions than the receiver, we show that the underlying game does not have any stable outcomes. In particular, the partitional equilibria of Crawford and Sobel (1982) are not stable. Hence, any learnt behavior must be changing over time. We then show using simulations that the typical pattern learnt in this environment is one where the algorithm divides the state space using a threshold. Above the threshold, the sender pools all of the states using the same message for all of those states. Below the threshold, however, the sender’s policy is constantly changing but very close to full revelation. For all of the currently fully revealing states, the state below will have an incentive to use the message associated with the state above it (due to the positive bias). In other words, the algorithm is constantly trying to inflate its private information below the threshold leading to stable cycling. Nevertheless, this constantly changing language is highly informative. Both the sender’s and the receiver’s welfare in our simulations is always above the best theoretical equilibrium from Crawford and Sobel (1982). Another interesting insight is that exploration, while necessary for learning, also hinders communication and can prevent the algorithm from ever learning to fully reveal, even in the no-bias case where full revelation maximizes payoffs. Because the sender experiments with non-negligible probability, the receiver understands that even in the best-possible scenario where the sender always truthfully reveals the state, any given message could have been generated from other states due to experimentation, so her choice will be more ‘cautious,’ as the posterior is pulled toward the prior. When the learning algorithm infers that the receiver does not fully follow its advice, it responds by adjusting messages to correct for these more cautious actions, which makes exaggeration profitable and leads the dynamics toward pooling at the extremes of the state space. In the no-bias environment, we argue that , for sufficiently high exploration rates, fully revealing policies are not stable under learning, and the simulations confirm that the long-run outcome is a highly but not fully informative pooling structure. Needless to say, our focus on tabular Q-learning raises external validity concerns. We nevertheless adopt it for three reasons. First, tabular Q-learning is a natural benchmark for our setting: it is widely used as a baseline method for learning optimal policies from payoff feedback. 1 1 1 Other natural candidates for this problem are contextual bandit algorithms; it turns out that Q-learning in this setting is equivalent to contextual versions of the ‘Exp3’ bandit algorithm ( 3 ) . Second, its transparency makes it suitable for isolating the mechanisms through which reward-driven updating shapes strategic communication. Finally, it is a foundational building block for more sophisticated reinforcement-learning architectures, so studying it allows us to isolate forces that plausibly carry over to broader classes of learning-based systems. 2. Related Literature Algorithmic advice from AI or machine learning

Read the complete text