Exploiting Expertise of Non-Expert and Diverse Agents in Social Bandit Learning: A Free Energy Approach
Erfan Mirzaei, Seyed Pooya Shariatpanahi, Alireza Tavakoli, Reshad Hosseini, Majid Nili Ahmadabadi · 2026
Read Exploiting Expertise of Non-Expert and Diverse Agents in Social Bandit Learning: A Free Energy Approach by Erfan Mirzaei, Seyed Pooya Shariatpanahi, Alireza Tavakoli, Reshad Hosseini, Majid Nili Ahmadabadi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-11757.txt.
Introduction
Mirzaei and co-authors study social bandit learning, where one agent learns partly by watching others act. The social agent sees others' actions but never their rewards, and no agent aims to teach. Their free energy method works over the policy space. It rates other agents' expertise with no oracle and no social norms to lean on. The agent then mixes its own experience with the policies it estimates from peers. The team proves convergence to the optimal policy.
Catalog record
| CREATOR | Erfan Mirzaei, Seyed Pooya Shariatpanahi, Alireza Tavakoli, Reshad Hosseini, Majid Nili Ahmadabadi |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 67271 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.11757
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Exploiting Expertise of Non-Expert and Diverse Agents in Social Bandit Learning: A Free Energy Approach Erfan Mirzaei, Seyed Pooya Shariatpanahi, Alireza Tavakoli, Reshad Hosseini, Majid Nili Ahmadabadi Abstract Personalized AI-based services involve a population of individual reinforcement learning agents. However, most reinforcement learning algorithms focus on harnessing individual learning and fail to leverage the social learning capabilities commonly exhibited by humans and animals. Social learning integrates individual experience with observing others’ behavior, presenting opportunities for improved learning outcomes. In this study, we focus on a social bandit learning scenario where a social agent observes other agents’ actions without knowledge of their rewards. The agents independently pursue their own policy without explicit motivation to teach each other. We propose a free energy-based social bandit learning algorithm over the policy space, where the social agent evaluates others’ expertise levels without resorting to any oracle or social norms. Accordingly, the social agent integrates its direct experiences in the environment and others’ estimated policies. The theoretical convergence of our algorithm to the optimal policy is proven. Empirical evaluations validate the superiority of our social learning method over alternative approaches in various scenarios. Our algorithm strategically identifies the relevant agents, even in the presence of random or suboptimal agents, and skillfully exploits their behavioral information. In addition to societies including expert agents, in the presence of relevant but non-expert agents, our algorithm significantly enhances individual learning performance, where most related methods fail. Importantly, it also maintains logarithmic regret. I Introduction Reinforcement learning (RL) in general, and bandit learning as a single-state RL in particular, is rooted in the study of animal and human learning [ 41 , 38 ] . However, humans and several animals, as social beings, employ a diverse set of social learning methods in harmony with RL. This type of learning is used to make optimal decisions in bandits and acquire complex behavior, resulting in faster adaptation to new tasks and environments, and to explore safer and more efficiently [ 3 , 24 , 10 ] . The complexity of human societies would not have been possible if they relied solely on individual learning to solve everyday decision-making problems [ 18 , 23 , 6 ] . Ignoring the social aspects of reinforcement learning algorithms can be a contributing factor to their poor performance compared to humans, particularly in simple tasks. Additionally, due to the rapid advances in human-in-the-loop technologies [ 29 , 7 ] and AI-based personalization of products and services, interactions between humans and artificial intelligence (AI) have become an integral part of daily life [ 42 ] . In this setting, we encounter societies of diverse artificial interactive learning agents that engage in complex bandit tasks. These factors present an opportunity to develop an effective combination of individual and social learning, addressing the main disadvantages of bandit learning, namely, high regret and slow learning. According to the theory of social learning, two different resources contribute to the learning of new tasks and skills: direct interaction with the environment and observation of others’ behavior and consequences [ 3 ] . The former is more closely related to reinforcement learning, while the latter is more associated with observational learning in the literature. This research focuses on applying social learning to bandit problems, which offers a simple yet elegant framework for studying the explore-exploit trade-off faced by an agent in an unknown environment. Our setting, known as Social Bandit Learning (SBL), closely resembles many real-world problems, including personal AI assistants and personal education systems. In personal education systems, each student interacts with an AI-powered tutor. These tutors can observe each other’s actions but not the rewards and other private data. The tutors can form a society while interacting with humans who may possess diverse expertise and utilities. The same is true for personalized recommender systems. These systems can form a society, too. By integrating social learning methods in parallel with individual learning, we can achieve lower regret and faster learning. Our setting involves a Social Agent (SA) that observes the actions of other agents, called individual agents (IAs), in the same environment, without access to their corresponding rewards and other private information. The agents independently pursue their own policy without any direct incentive to teach one another, and we make no assumptions about their expertise or the relevance of their policy to our SA’s task. Therefore, in the absence of any social evaluations and norms and due to reward privacy, the SA should mainly evaluate other agents’ relevance as well as the levels and scope of their expertise based on a self-referenced evaluation [ 45 , 44 ] . The challenge here is that, due to a lack of sufficient experience in early trials, typical measures for self-referenced evaluation are not accurate enough in the early stages of learning. This leads to ignoring social learning or high regret in the early trials. Our solution to this challenge is to include global and absolute measures for assessing the suitability of information besides self-referenced evaluation. The next challenge for the SA is how to integrate its own direct experiences (action and reward) and the observed behavior data of others to enhance its policy and reduce its regret. In this research, we propose a free energy solution in the policy space that resolves the two mentioned challenges in a unified method. Free energy minimization models the behavior of bounded-rational decision-makers, aiming to strike a balance between expected utility and information-processing cost [ 33 ] . The free energy in our method includes the entropy of candidate behavior policy as a measure of suitability for the observed behavior and the divergence of candidate policies from a reference policy as a self-referenced evaluation measure. Our approach differentiates from other multi-agent reinforcement learning methods by not assuming that agents can share private information, such as rewards, observations, or gradients, during training. Such assumptions are unlikely to hold in real-world social learning scenarios that involve competing agents, agents with intrinsic motivations, and agents from different companies. Importantly, these assumptions may not generalize to human-in-the-loop or human-AI interaction scenarios [ 22 , 17 ] . It is worth noting that this problem should be distinguished from other multi-agent bandit problems, specifically cooperative bandits, where several agents share information about actions and rewards obtained [ 25 ] . Our method inherently includes imitative learning and observational learning but it is neither limited to the presence of an expert nor to sharing a common utility function among the agents. In the next section, we review the literature related to social learning in the RL bandit framework. We then present the problem statement and the assumptions. For the sake of completeness, we explain the TS policy estimation as well as the free energy model of decision-making and mention the individual learning methods employed in this research. Subsequently, we describe our method and demonstrate its convergence to the optimal policy. The experimental results in various scenarios are presented next. Lastly, we analyze the empirical findings, discuss their implications and limitations, and propose promising research directions. II Related Work Learning from other agents, whether artificial or human beings, has been extensively studied across diverse research domains. In complex robotics tasks, imitation learning has gained popularity.