Evolution of Collective AI Beyond Individual Optimization

Ryosuke Takata, *, Yujin Tang, Yingtao Tian, Norihiro Maruyama Hiroki Kojima Takashi Ikegami · 2024

Read Evolution of Collective AI Beyond Individual Optimization by Ryosuke Takata, *, Yujin Tang, Yingtao Tian, Norihiro Maruyama Hiroki Kojima Takashi Ikegami (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-02085.txt.

Introduction

Ryosuke Takata and co-authors evolved simple agents and watched group behavior decay as single agents improved in 2024. Each agent is a small neural network driving a vehicle that follows scent trails. Clones of one evolved champion formed the test groups.

Evolution first split the population into distinct types. Group fitness then fell in later stages even as single performance stayed high and networks grew simpler. The slide tracked a loosening link between sensors and motors in the champions. Agents tuned hard for solo success, the authors argue, almost always make worse teammates. The study probes which evolutionary paths let group-level skill survive individual tuning.

Catalog record

CREATORRyosuke Takata, *, Yujin Tang, Yingtao Tian, Norihiro Maruyama Hiroki Kojima Takashi Ikegami
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-02085

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Evolution of Collective AI Beyond Individual Optimization Ryosuke Takata, *, Yujin Tang, Yingtao Tian, Norihiro Maruyama Hiroki Kojima Takashi Ikegami Abstract This study investigates collective behaviors that emerge from a group of homogeneous individuals optimized for a specific capability. We created a group of simple, identical neural network based agents modeled after chemotaxis-driven vehicles that follow pheromone trails and examined multi-agent simulations using clones of these evolved individuals. Our results show that the evolution of individuals led to population differentiation. Surprisingly, we observed that collective fitness significantly changed during later evolutionary stages, despite maintained high individual performance and simplified neural architectures. This decline occurred when agents developed reduced sensor-motor coupling, suggesting that over-optimization of individual agents almost always lead to less effective group behavior. Our research investigates how individual differentiation can evolve through what evolutionary pathways. K eywords Collective behavior \cdot Artificial intelligence \cdot Neuroevolution \cdot Chemotaxis Figure 1: Neural network model of an agent. The agent receives input from six sensors (five on the periphery of its circular body and one at the center) and moves using two motor outputs (linear and angular velocities). The agent’s neural network consists of 6 sensor input neurons, 6 hidden neurons, 2 output neurons, and 2 context neurons, with a bias neuron in each layer. 1 Introduction Artificial Intelligence (AI) has witnessed significant advances with the emergence of powerful neural network (NN) models. Examples include large language models [ 1 ] and image generation models such as DALL-E [ 2 ] , Imagen [ 3 ] , and Parti [ 4 ] . Each has achieved previously unseen capabilities as powerful individuals through recent technical breakthroughs. On the other hand, the biological evolutionary strategy focuses more on the direction of collective intelligence compared to individual ability, especially for species living in populations [ 5 ] . Unlike individual intelligence, which deals with challenges independently, collective intelligence necessitates the ability to process information, operate in a decentralized manner, and adaptively integrate information based on context. This distinction is evident in social insects, such as ants and bees, where collective behavior with role differentiation emerges not from highly complex individuals but through simple interactions among members. Recent studies have demonstrated various approaches to understanding collective behavior. Neural controllers evolved to climb environmental gradients can develop exploration-exploitation strategies [ 6 ] , while effective gradient climbing can emerge from local sensing and interactions without global planning [ 7 ] . The exploration processes themselves show remarkable parallels between environmental and internal memory search, suggesting common evolutionary origins [ 8 ] . Social interactions shape exploration strategies that may not necessarily optimize group-level efficiency [ 9 ] . Physical approaches to collective behavior in insects have revealed how individual properties and their interactions lead to various scales of collective patterns [ 10 ] . These findings suggest that collective intelligence emerges through the complex relationship between individual and group-level behaviors [ 11 ] , where local interactions and environmental feedback play crucial roles. We seek to bridge the biological collective intelligence with artificial intelligence, which we denote as “Collective AI” [ 12 ] and [ 13 ] , focusing on the use of a population of neural networks. Here the intelligence of Collective AI is embodied not in the capacity of individual neural network, but instead in how each member (or agent) of the population interacts with each other, as well as the dynamics of the resulting population. This interaction can be quantified through information-theoretic measures, particularly focusing on how environmental information is processed and integrated with internal states to generate collective behaviors. In this study, we focus on how individual optimization of a specific behavior–chemotaxis–can lead to emergent collective behavior when agents interact in a shared environment. Specifically, we investigate the evolution of collective intelligence in populations of neural network-controlled agents inspired by clonal insects. These agents, evolved individually to perform chemotaxis, interact in multi-agent simulations where communication occurs via pheromone signals. In our simulation, we observe that collective intelligence is realized, which accompanies role differentiation. Individuals that specialize in collecting chemicals in the environment can effectively use the same chemicals for communication when they are in a group. While individual evolution prioritizes optimization for specific tasks, we seek to understand how group-level dynamics emerge and diverge from individual optimization. 2 Models In the model, we first induce chemotaxis in a single agent: in the framing of chemotaxis, this agent then senses the chemicals produced by other individuals, using this chemotactic response to gather with them. The resulting collective behavior is subsequently analyzed. 2.1 Agents Controlled by Neural Networks An agent’s behavior is controlled by a neural network (Figure 1 ), the parameters of which changes through an neuroevolution process. The neural network has an input layer of 9 neurons ( 1 bias, 6 input neurons, 2 context neurons), a hidden layer of 7 neurons ( 1 bias, 6 hidden neurons), and an output layer of 4 neurons ( 2 output neurons, 2 context neurons). Neuroevolution was performed on all 82 weights of the neural network. We used CMA-ES [ 14 ] in EvoJAX [ 15 ] as our evolutionary algorithm. CMA-ES generates multiple candidate solutions using a multivariate normal distribution and calculates their goodness of fit. The advantage here is that it is easy to parallelize the evaluation of the objective function with the number of candidate solutions. The agent moves by having its velocity and angular velocity determined by a neural network. The range of values for the angular velocity \omega(t) is [-0.05,0.05] , and the following formula is used to update the agent’s angle \theta(t) . \theta(t)=\theta(t-1)+\omega(t) (1) The range of values for velocity v(t) is [0.0,1.0] , and the following formula is used to update the agent’s position x(t) and y(t) . Here, the field has periodic boundary conditions. x(t)=x(t-1)+v(t)\cdot\cos(\theta(t)) (2) y(t)=y(t-1)+v(t)\cdot\sin(\theta(t)) (3) 2.2 Experimental Setup The experiment consists of two phases: (1) an evolution phase where a single agent’s neural network is trained to acquire chemotaxis behavior, and (2) a test phase where the evolved neural network is replicated across 1024 agents for evaluation. In the test phase, evaluation is conducted on a homogeneous population where all agents share identical neural network parameters. We evaluate our evolved neural network periodically. Concretely, for every 10 generations, we pick the best individual from the population and replicate its parameters (i.e. the synaptic weights) 1024 times to create identical agents, and use these agents to conduct a multi-agent simulation. The parameter values used in the simulations are detailed in Table 1 . The environmental parameters, such as spatial dimensions and pheromone evaporation rate, remain identical between the single-agent evolution phase and the multi-agent test phase. Table 1 : Summary of the simulation parameters. Parameter Value Field width 600 Field height 600 Agent body radius 20 Agent sensor radius 2 Number of sensors on agent 6 Number of agent actions 2 Number of initial pheromone spots 5 Pheromone decay rate 0.001 Maximum number of steps (for a single agent) 1000 Maximum number of steps (for multi agent) 5000 Maximum generations 2000 Population 100 Initial standard deviation of candidate solution individuals 0.1 2.3 Pheromone Environment The evolution phase which agents are subjected to chemotaxis, the environment

Read the complete text