Improving How Agents Cooperate: Attention Schemas in Artificial Neural Networks
Kathryn T. Farrell, Kirsten Ziman, Michael S. A. Graziano · 2024
Read Improving How Agents Cooperate: Attention Schemas in Artificial Neural Networks by Kathryn T. Farrell, Kirsten Ziman, Michael S. A. Graziano (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2411-00983.txt.
Description
Abstract Growing evidence suggests that the brain uses an “attention schema” to monitor, predict, and help control attention. It has also been suggested that an attention schema improves social intelligence by allowing one person to better predict another. Given their potential advantages, attention schemas have been increasingly tested in machine learning.
Catalog record
| CREATOR | Kathryn T. Farrell, Kirsten Ziman, Michael S. A. Graziano |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 41 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2411-00983
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Improving How Agents Cooperate: Attention Schemas in Artificial Neural Networks Kathryn T. Farrell, Kirsten Ziman, Michael S. A. Graziano Abstract Growing evidence suggests that the brain uses an “attention schema” to monitor, predict, and help control attention. It has also been suggested that an attention schema improves social intelligence by allowing one person to better predict another. Given their potential advantages, attention schemas have been increasingly tested in machine learning. Here we test small deep learning networks to determine how the addition of an attention schema may affect performance on a range of tasks. First, we found that an agent with an attention schema is better at judging or categorizing the attention states of other agents. Second, we found that an agent with an attention schema develops a pattern of attention that is easier for other agents to judge and categorize. Third, we found that in a joint task where two agents paint a scene together and must predict each other’s behavior for best performance, adding an attention schema improves that performance. Finally, we find that the performance improvements caused by an attention schema are not a non-specific result of an increase in network complexity. Not all performance, on all tasks, is improved. Instead, improvement is specific to “social” tasks involving judging, categorizing, or predicting the attention of other agents. These results suggest that an attention schema may be useful in machine learning for improving cooperativity and social behavior. Introduction Growing evidence suggests that the brain uses an “attention schema” to monitor, predict, and help control attention [1-5]. The attention schema may be used to model more than one’s own attention. It may also be directed to model the attention of others, affording better theory of mind, better predictions of the behavior of others, and therefore more effective social interaction [1,6-12]. In the present study, we ask whether adding an attention schema to an artificial agent improves its ability to make judgements about the attention of other agents, improves its ability to have its own attention judged by others, and improves its ability to coordinate with another agent in a simple joint task. Recently, versions of attention schemas have been incorporated into artificial neural networks to test whether the same principles apparent in the human brain might provide benefits in machine learning, and to determine whether lessons from the more controlled, artificial case might provide conceptual insights into human cognition [13-17]. Two studies in particular suggest that self-models may improve performance in cooperative tasks. In one study, Liu et al. [15] trained interacting groups of networks on visual tasks that required the agents to coordinate with each other. A transformer-style visual attention was used. In some conditions, networks were provided with an attention schema, in the form of a learned ability to predict the state of their own attention mechanism. With an attention schema, agents performed the social coordination tasks better. One interpretation is that when an agent is trained to predict its own attention states, it also becomes better at predicting the attention states of other agents in the same environment. In that hypothesis, learning to self-model leads to an improvement in the ability to model others and thus to an improvement in group task performance. In a second recent study on the effects of self-modeling, Premakurma et al. [16] trained artificial networks on a variety of discrimination tasks, including both visual and linguistic tasks. In some conditions (the self-model conditions), the agents were also trained on an auxiliary task in which they were rewarded for predicting the activation states of their own inner layers. The networks that were trained on the auxiliary task showed a consistent change. They learned to make themselves simpler, more regularized, and more parameter-efficient as they performed the primary task. These gains were found across all types of networks studied. It was hypothesized that when a network is given the auxiliary task of predicting its own internal activity states, one way for it to ensure better performance is to learn to make its internal states simpler and more regular. The simpler the internal states of the network, the easier they are to predict. Therefore, as the network learns to predict itself, it learns to simplify itself. According to that hypothesis, learning a self-model restructures the self, regularizing the network, and making internal states an easier target of predictive modeling. Based on these prior findings, we proposed the following two hypotheses about the relationship between self-models and social behavior. First, when an agent learns to self-model, such as when it learns an attention schema, it can transfer some of that learning toward predictively modeling other agents. Predictive modeling of other agents then becomes a useful part of interacting with them. Second, when an agent learns to self-model, it restructures itself to be more regularized and more predictable, and therefore the agent becomes a more effective target for being predictively modeled by other agents. Both changes – becoming better at modeling others, and becoming a more transparent target for being modeled by others – should render the agent a more effective member of a cooperative group. The purpose of the present study was to test these hypotheses about how self-modeling, and in particular attention schemas, might influence multi-agent interaction. We conducted three related experiments. Experiment 1 asked: Do attention schemas improve the ability of networks to model each other’s attention states? Experiment 2 tested an alternative explanation: Are the performance benefits of an attention schema merely a non-specific effect of added network complexity? Finally, Experiment 3 addressed the question of multi-agent cooperation: Do agents with attention schemas perform better on a simple joint task? Experiment 1: Do attention schemas improve the ability of agents to model each other’s attention states? Introduction to Experiment 1 The purpose of Experiment 1 was to train artificial agents with and without attention schemas and to test two hypotheses. Hypothesis 1 was that an agent with an attention schema will be better at learning to make categorical judgements about the attention of other agents. Hypothesis 2 was that when an agent has an attention schema, other agents will perform better at learning to make categorical judgements about its attention. Agents were first trained on a visual categorization task (e.g. categorize a picture as showing a golf ball or a garbage truck). We then froze the weights in all layers except a final block, and trained the agents on a new task, the “attention judgement” task. The task allowed us to test the ability of one agent to make categorical determinations about the attention states of another agent. The attention judgment task was modified from Ziman et al. [18], who demonstrated the ability of people to assess the attention states of others. In the human version of the task, participants looked at a spotlight of attention moving around a picture and judged whether the spotlight represented real attention (based on eye movements recorded previously from a real person), or whether it represented a randomly scrambled version of attention (the recorded attention locations presented in a random order). Since people were able to perform above chance, the human brain must contain information about the typical or predicted patterns of attention and must be