Agent for User: Testing Multi-User Interactive Features in TikTok
Sidong Feng, Changhao Du, Huaxiao Liu, Qingnan Wang, Zhengwei Lv, Gang Huo, Xu Yang, Chunyang Chen · 2025
Read Agent for User: Testing Multi-User Interactive Features in TikTok by Sidong Feng, Changhao Du, Huaxiao Liu, Qingnan Wang, Zhengwei Lv, Gang Huo, Xu Yang, Chunyang Chen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-15474.txt.
Introduction
Sidong Feng and co-authors send AI agents to test TikToks multi-user features in a 2025 paper. Live streams and voice calls need several people acting at once, which strains ordinary app testing. Their answer is a farm of virtual devices, each steered by a language model playing a user.
The agents coordinate taps and talk to run shared scenarios end to end. Across twenty-four tasks in the real app, the setup covered three quarters of the features with actions closely matching human testers. Developer time fell sharply, and the system joined TikToks own platform. It has since caught twenty-six multi-user bugs in production use.
Catalog record
| CREATOR | Sidong Feng, Changhao Du, Huaxiao Liu, Qingnan Wang, Zhengwei Lv, Gang Huo, Xu Yang, Chunyang Chen |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 51 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-15474
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Agent for User: Testing Multi-User Interactive Features in TikTok Sidong Feng, Changhao Du, Huaxiao Liu, Qingnan Wang, Zhengwei Lv, Gang Huo, Xu Yang, Chunyang Chen Abstract TikTok, a widely-used social media app boasting over a billion monthly active users, requires effective app quality assurance for its intricate features. Feature testing is crucial in achieving this goal. However, the multi-user interactive features within the app, such as live streaming, voice calls, etc., pose significant challenges for developers, who must handle simultaneous device management and user interaction coordination. To address this, we introduce a novel multi-agent approach, powered by the Large Language Models (LLMs), to automate the testing of multi-user interactive app features. In detail, we build a virtual device farm that allocates the necessary number of devices for a given multi-user interactive task. For each device, we deploy an LLM-based agent that simulates a user, thereby mimicking user interactions to collaboratively automate the testing process. The evaluations on 24 multi-user interactive tasks within the TikTok app, showcase its capability to cover 75% of tasks with 85.9% action similarity and offer 87% time savings for developers. Additionally, we have also integrated our approach into the real-world TikTok testing platform, aiding in the detection of 26 multi-user interactive bugs. I Introduction Fig. 1: Illustration of LIVE together. TikTok 1 1 1 https://www.tiktok.com/en/ , also known as Douyin in China, commands a global user base with over 1.04 billion monthly active users, becoming one of the most popular social media apps in the world. It offers a diverse set of features, including voice calls, live streaming, and video conferencing, which have transformed the way users engage with each other, facilitating immediate communication, collaboration, and social connection. With TikTok’s rapid expansion and introduction of new features, app quality assurance has become both increasingly crucial and challenging. Feature testing is an indispensable approach. This involves developers writing automated scripts tailored to specific functional task objectives, with a focus on covering the testing of the app’s primary capabilities. Nevertheless, testing features within the TikTok app poses distinct challenges. Our empirical analysis of 7,870 feature testing tasks within TikTok, as discussed in Section II , reveals that 40.31% of these tasks pertain to the interactions between multiple users. For instance, as depicted in Fig. 1 , a user (User 1 ) initiates a multi-guest LIVE session by inviting friends (User 2 ) to join a collaborative communication. Despite their importance, testing these multi-user interactive features can be exceptionally time-consuming and laborious for developers, due to two major challenges. First, the task automation test scripts must be adaptable across different devices, a requirement that becomes more complex when it involves the variety of devices used in testing multi-user interactive features. Second, the sequence of interactions between multiple devices requires precise coordination. Any deviation from the intended sequence, such as attempting to accept before an invitation is properly sent, could lead to the feature failing to trigger correctly. While extensive research [ 1 , 2 , 3 , 4 , 5 ] has been devoted to automated app testing, none have adequately covered the testing of those collaborative and sequential multi-user interactive features across multiple devices. Recent studies [ 6 , 7 ] have proposed the use of agent frameworks in the domain of software testing. In these frameworks, agents are programmed to autonomously perceive and interact with their environment, working together to solve problems. However, these frameworks are often constrained by hard-coded rules tailored to specific software, which limits their adaptability to apps like TikTok with dynamic GUI contents and flexible user interactions. In recent years, the rise of advanced Large Language Models (LLMs) has introduced a paradigm shift in this space. Unlike traditional rule-based agents, LLM-powered agents can adapt to diverse software environments with human-like flexibility. A notable example is Anthropic’s release of Claude 3.5 Computer Use [ 8 ] , introducing an LLM-powered GUI agent designed for interacting with software interfaces in a more human-like and adaptive manner. Inspired by the human-like capability of LLMs, we propose a novel multi-agent framework powered by the LLMs to autonomously conduct multi-user interactive feature testing. This framework employs agents that simulate users interacting with mobile devices, working collaboratively to trigger multi-user interactive features. Specifically, taking a description of a multi-user interactive task as input, we first build a virtual device farm to support a number of virtual devices to allocate. Following this setup, we distribute the task to individual agents, each representing a user and paired with a virtual device. These agents, driven by LLMs, mimic user interactions on their respective devices and autonomously guide their navigation through the GUI screens. By operating in a collaborative manner, the agents contribute to the automation of multi-user interactive tasks. To evaluate the performance of our approach, we first carry out a pilot experiment involving 24 interactive tasks in the TikTok app. Results demonstrate that our approach achieves a success rate of 75% and 85.9% of its actions are the same as the ground truth, significantly outperforming three state-of-the-art baselines. Additionally, our approach offers a time-saving advantage, reducing the time required for multi-user feature testing by 87% for professional developers. Beyond the effectiveness of the approach, we also assess its practicality in the real-world environment. We integrate our approach with the TikTok testing platform, triggering it whenever new functionalities need testing or a new version is released, accompanied by task descriptions. In a real-world platform involving the testing of 3,318 multi-user interactive tasks, our approach is instrumental in uncovering 26 interactive bugs, indicating the potential value of our approach in enhancing the practical testing process. The contributions of this paper are as follows: • We discover the problem of multi-user interactive features within the software. • We propose a novel approach, that harnesses a multi-agent LLMs framework for the autonomous automation of multi-user interactive tasks. • Comprehensive experiments, including the effectiveness and usefulness evaluation of our approach for interactive task automation. Additionally, we incorporate our approach into a real-world testing platform to illustrate its practicality. II Background & Motivation In this section, we first define the problem of multi-user interactive features. Then, we briefly discuss the practical challenges of testing these interactive features in real-world testing environments, underscoring the necessity for specialized tool support. II-A Definition of Multi-User Interactive Feature Definition : Multi-user interactive feature is a component of software that enables 1 multiple users to 2 interact with each other in 3 nearly real-time within a software. The definition of multi-user interactive features is dissected into three fundamental elements. First, the feature enables users to interact with multiple users through the software. Note that interactions between a user and the software or the administrator are not categorized under multi-user interactive features. An example of this would be the process of activity organization, where an end-user books a reservation and the admin confirms it. Such interactions are predominantly administrative and do not embody the peer-to-peer collaborative or communicative intent that is the hallmark of multi-user interactive features. The second critical aspect of the interactive feature is its fundamental interactivity. This characteristic is marked by the presence of interactive cues including notifications or pop-up messages, that differ significantly from collaborative functions like text editing.