Video games help push the boundaries of AI Authors: Stephen Ornes Published: 2026 A few years ago, a group of computer scientists set 25 players loose in Smallville, a custom-made virtual “sandbox world”—an open-ended video game like Minecraft that allows players to freely explore and interact with each other. The catch: The players in Smallville were all virtual, each driven by its own large language model, or LLM. The goal, says Joon Sung Park, who led the project while finishing his PhD at Stanford University in California, was to see what kinds of human-like behaviors evolved. In one recent project that used video games to explore artificial intelligence, computer scientists introduced 1,000 AI-driven agents into the game Minecraft and watched as an AI civilization emerged. It included artists, chefs, and explorers. Image credit: Shutterstock/mkfilm. To get the players started, Park and his colleagues wrote a paragraph describing the identity of each. For example, for one they wrote, “John Lin is a pharmacy shopkeeper at the Willow Market and Pharmacy who loves to help people.” What happened next was up to the LLMs. In the spring of 2023, the researchers reported on what unfolded over two game days in Smallville (1). The game’s AI populace had been surprisingly busy with familiar-seeming tasks. “They would wake up in the mornings, go to work, and do all kinds of things,” Park says. When one agent was instructed (by a researcher on Park’s team) to throw a Valentine’s Day party with no further instructions, it issued invitations and began party preparations. Some Smallvillians began asking dates to the party and making plans to get there on time. When one agent told another that it planned to run for mayor, the campaign quickly became a hot topic in Smallville as agents discussed how they planned to vote. The sandbox environment gave Park a way to explore human behavior using LLMs; his study is one of the latest in a long history of using games to push computer technology forward. Some scientists develop games to explain difficult concepts, or even to crowdsource new ideas in a field (2). “When it comes to developing artificial intelligence, games were there from the beginning,” says Julian Togelius, a computer scientist and video game researcher at New York University who explores the intersection of AI and video games. Games like chess and Go, he says, gave early researchers a way to test the capabilities of new AI tools—and remain valuable touchstones for new insights (3). Video games offer expansive opportunities for probing AI, Park says. LLMs are trained on broad datasets that include information found online, from social media feeds to message boards to Wikipedia, but they don’t learn from real-world interactions. The open environment of video games can show situations in which AI agents, through interactions, make convincingly human-like decisions. Instead of just using their individual training data and memory, Park says, perhaps models could improve by learning through social interactions. Once researchers offered up starting conditions for agents in their Smallville game, those players began taking on roles, engaging in activities, and planning events. One even decided to run for mayor. Image credit: Joon Sung Park. Society Simulation Already, video games offer a way to probe the unexpected and myriad ways that AI can evolve. In 2019, a few years before the debut of ChatGPT, OpenAI reported that AI agents in an open-source game world developed surprising strategies, well outside their training, for winning at hide-and-seek (4). In 2022, SONY developed an AI-trained driver named GT Sophy that bested the top human players in the game Gran Turismo (5). The game’s standard computer drivers help ordinary players practice, but GT Sophy learned the course through repeated races and crunching physics data. For his part, Park didn’t initially set out to use video games. His research interests were more focused on behavior and interaction than on the limits of AI. Starting about five years ago, well before the Smallville experiment, he and his colleagues, in the Stanford lab of computer scientist Michael Bernstein, wanted to create credible simulations of human societies. Such simulations, he says, would enable developers and researchers to predict how real people would behave in virtual environments. A developer building a new social media platform, for example, may want some way to anticipate the diverse ways in which throngs of users could interact. Policymakers may want to predict how a diverse population might react to public health campaigns and messages; product developers may want to know how a population will respond to a new product launch. “When there’s like a hundred people, a thousand people,” Park says, “it gets really confusing because of all the emergent behaviors that might happen.” LLMs, designed to respond to queries from users, suggested a way to offer better simulations of societal interactions. “We had this idea,” he says, “that if the linked models can actually replicate human behavior, then maybe we can start prototyping social dynamics.” For a paper published in 2022, Bernstein’s group used LLMs to generate thousands of virtual, interacting AI members in an online community, something akin to the discussion threads found on Reddit (6). The researchers wanted to “help people better understand the social dynamics of different subreddits that they might want to create,” Park says. Encouraged by those results, Park and his group then began to think about other virtual environments. It didn’t take long to recognize the potential of video games, he says. Most of the researchers had played games such as Minecraft or Pokémon Go when they were younger. Those games are populated by large numbers of nonplayer characters, or NPCs, often equipped with built-in tasks or behaviors. Some game developers have explored using AI to build NPCs that populate a game, but “that is actually very, very hard to do,” says Togelius, both because they have hefty computational requirements and because they’re unpredictable enough that they can disrupt or completely derail the flow of a story within a game. Park’s intent was different. He wanted to know if LLM-driven NPCs could plausibly mimic human social interactions. “What if we can create NPCs basically by leveraging this ability to generate human behavior that can basically live the way we might?” Park asks. “We didn’t quite know what it was going to be useful for, but we thought it was going to be really, really cool if we could do it right.” “We didn’t quite know what it was going to be useful for, but we thought it was going to be really, really cool if we could do it right.” —Joon Sung Park Which led them to Smallville. “In a game, spaces replicate human space really well, right?” Park says. “It’s not perfect, but it is sort of a physical dimension.” The sandbox world offered a way to test what would happen if it were populated only by NPCs whose actions were not deterministic but instead generated by AI. Interactions like the campaign rumors and the Valentine’s Day party were surprising, Park says, because they seemed natural and human-like. And such results offered support for the idea of using AI as a simulator of social situations. Since then, the researchers have expanded that work. For a paper posted to arXiv in late 2024 (7), the group interviewed 1,000 people, then used their responses to train AI agents, one for each participant. Later, both humans and their AI doppelgangers completed attitude and behavioral surveys commonly used in psychology studies. Responses from these “generative agents” to questionnaires matched the real people’s responses 85% of the time (7). Other groups have been pushing different sorts of AI–video game combinations. In the fall of 2024, researchers at Altera, an AI startup, reported that they’d plugged up to 1,000 AI-driven agents—the most Minecraft could handle—at a time into the game. They became artists, chefs, and explorers as a kind of artificial AI civilization took shape (8). The Stanford team has pushed their work forward, too. In February 2026, Park, Bernstein, and their colleagues launched Simile.ai, a simulation of human society in which the AI agents are digital twins of real people. Virtual Fight Not all groups are looking at sandboxes for inspiration; some are looking at games like Street Fighter, a classic two-player arcade game that, unlike the cooperative environment of Minecraft, sets players against each other. In October 2023, computer scientists at the Singapore University of Technology and Design reported that they’d trained an AI model to play Street Fighter through reinforcement learning. (This approach, in which a model improves performance through trial and error, is similar to how researchers trained AI models to excel at games like chess and Go.) After two days of training, the AI player could readily defeat the best human players, having taught itself the most strategic moves. In March 2024, programmers at a hackathon sponsored by Mistral AI, a French technology company, wrote code that allowed two of Mistral’s LLMs to fight each other, in real time, in Street Fighter. Then, a developer named Banjo Obayomi, at Amazon Web Services (AWS), decided to push that test further by writing code to corral 14 AI models to fight each other (9). The AI models duked it out in more than 300 one-on-one matches, using LLM-generated text commands translated into virtual physical actions (9). Obayomi ranked the players, each representing an LLM, using a metric similar to chess rankings. He says he was surprised by the diversity of strategies used by the bots. “All of them had different types of outputs,” he says. “I was interested to see how they can be given the same kind of task but approach it differently.” The trials brought out unexpected behaviors akin to “hallucinations,” nonsensical or incorrect responses to queries by LLMs. Some models gave directions to execute things like “special move,” “jump cancel,” and “hardest hitting combo of all,” none of which are legitimate Street Fighter options. One of the models, Claude 2.1, wouldn’t even engage. “I apologize,” it said instead. “Upon reflection I do not feel comfortable recommending violent actions or strategies, even in a fictional context.” At the end of the punching, jumping, and kicking, Claude 3 Haiku, a model from Anthropic, came out on top. Claude 3 Haiku is a small language model, which means it has fewer parameters—the internal knobs that can change through training to adjust the strength of the connections within the model—thus improving performance. Haiku’s welterweight size likely gave it an advantage, Obayomi says. In general, in the tests, he says, smaller models performed faster, and more capably, than larger ones. Obayomi suspects that even smaller models might be lither, and if they’re better at video games, maybe they would excel at other tasks, too. “How small can you go for the model to understand this concept and understand the game?” he asks. Smaller models consume less energy and can perform better on specific tasks, and video games may help researchers better recognize those tasks. The Street Fighter matches also suggest video games might be used for real-time benchmarking—tests that help reveal the strengths and limitations of algorithms and models. Street Fighter, for example, might show what kinds of tasks lead to longer response times. Minecraft, meanwhile, could demonstrate dynamic behavior traits researchers can try to cultivate in AI agents. There are plenty of historical precedents for getting AI to play games. Decades ago, chess and Go were used to test early efforts at AI. In a 1950 paper, Claude Shannon used chess to guide the development of new algorithms for early computers (10). Monte Carlo Tree Search, an algorithm that revolutionized how computers play Go, has become a useful decision-making tool in complex environments. In the early 1990s, computer scientist David Chapman devised a system called Sonja that could visually interpret and play a video game—called Amazon—on its own, even taking instructions from humans (11). With the rise in popularity and power of LLMs, researchers have used benchmarking both to figure out what generative AI models can do and to develop strategies for making them better. Large benchmarking efforts have involved discrete tasks like adding numbers, translating languages, or answering factual questions. Benchmarks used to challenge LLMs such as GPT-4 or PaLM include assignments like multiplying numbers or generating computer code (12). Benchmarks have shown that LLMs can complete tasks well beyond their design, which is to offer answers to prompts by predicting the next word in a sentence. Most of these tasks are static and untimed, though. “The speed element is missing in some traditional benchmarks,” Obayomi says. Video games, he says, might show which models can act and respond the fastest, not just the most accurately. As LLMs like those from Google or OpenAI became more powerful and popular, testing them on video games seemed like an obvious way to see their limits, Togelius says. But at the same time, he sees the work from AWS as more of a clever demonstration than a meaningful insight into the capabilities of LLMs, partly because the reported results only focus on the winner. “It’s fun, but it’s just not finished,” he says. “They don’t have anything like statistical significance” in their conclusions. “What it really just highlights is how slow all these [tested] models are.” Other AI approaches, such as reinforcement learning or computer vision, might produce interesting insights through video games, he says. But for Togelius, LLMs seem misaligned with the tasks of a game like Street Fighter because LLMs focus on language, and Street Fighter requires quick actions. The Amazon researchers aren’t deterred. “It’s still really early, but there’s a focused approach to leveraging AI tools in game development,” says Chris Melissinos, a video game industry expert who helped Obayomi coordinate the project. Focused research into how agents behave in dynamic, interactive environments inspires new ways to use AI for building worlds and NPCs. “There’s a lot of experimentation going on,” he says—how AI can improve game play, and where it falters. “It’s like any new kit you get,” he adds. “How can I break this thing? How can I push it to its edge?”