MindFlow: Revolutionizing E-commerce Customer Support with Multimodal LLM Agents

Ming Gong, Xucheng Huang, Chenghan Yang, Xianhan Peng, Haoxin Wang, Yang Liu, Ling Jiang · 2025

Read MindFlow: Revolutionizing E-commerce Customer Support with Multimodal LLM Agents by Ming Gong, Xucheng Huang, Chenghan Yang, Xianhan Peng, Haoxin Wang, Yang Liu, Ling Jiang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-05330.txt.

Introduction

Ming Gong and co-authors present an open AI helper for online shop support in this 2025 paper. Shop chats mix text, images, and order data, which strains plain chatbots. Their agent joins memory, choice, and action parts in one frame, and treats a vision model as a tool it can call.

Live A/B tests and lab ablations grade the agent on hard mixed queries. Shopper joy rose and running costs fell in the live tests. The team reports a large gain over the prior setup in real use. Taking parts away in lab tests shows each part earns its place. The code is open, so other shops can adapt the agent.

Catalog record

CREATORMing Gong, Xucheng Huang, Chenghan Yang, Xianhan Peng, Haoxin Wang, Yang Liu, Ling Jiang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE23 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-05330

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

MindFlow: Revolutionizing E-commerce Customer Support with Multimodal LLM Agents Ming Gong, Xucheng Huang, Chenghan Yang, Xianhan Peng, Haoxin Wang, Yang Liu, Ling Jiang Abstract Recent advances in large language models (LLMs) have enabled new applications in e-commerce customer service. However, their capabilities remain constrained in complex, multimodal scenarios. We present MindFlow , the first open-source multimodal LLM agent tailored for e-commerce. Built on the CoALA framework, it integrates memory, decision-making, and action modules, and adopts a modular “MLLM-as-Tool” strategy for efficient visual-textual reasoning. Evaluated via online A/B testing and simulation-based ablation, MindFlow demonstrates substantial gains in handling complex queries, improving user satisfaction, and reducing operational costs, with a 93.53% relative improvement observed in real-world deployments. The code will be released upon publication to support future research. 1 Introduction The exponential growth of e-commerce has placed increasing demands on customer service systems , which are now expected to resolve complex multimodal inquiries in real time while maintaining high user satisfaction ( Gajewska et al. 2020 ) . Despite recent progress in LLMs, significant challenges remain in real-world deployment ( Chaturvedi and Verma 2023 ; Ren et al. 2024 ) . Current systems commonly struggle with three key challenges: integrating visual and textual inputs effectively, maintaining contextual coherence across multi-turn dialogues, and making adaptive decisions in dynamic, open-ended scenarios. These shortcomings collectively result in lower resolution rates, higher operational costs, and degraded user experience. To address these challenges, we propose MindFlow, a novel multimodal LLM agent framework designed specifically for e-commerce customer service. Built on the CoALA architecture Sumers et al. 2023 , MindFlow tightly integrates a memory module, a decision-making module and an action module to generate precise, context-aware responses. It further adopts the “MLLM-as-Tool” paradigm, which decomposes visual-semantic tasks and leverages Multimodal Large Language Models (MLLMs) for more efficient visual-textual reasoning in complex multimodal interactions. We conduct comprehensive evaluations through online A/B testing and simulation-based ablation studies. Real-world online A/B testing demonstrates significant gains over traditional rule-based systems, with reduced latency and operational costs. Together with extensive simulation-based ablation studies, these results underscore the effectiveness of our decision-making and action modules, and highlight the robustness of the “MLLM-as-Tool” paradigm for multimodal reasoning in practical deployments. Our primary contributions are: • First open-source multimodal LLM agent for industrial e-commerce , providing a practical framework with tightly integrated memory, decision-making, and action modules for real-time, context-aware reasoning. • Modular “MLLM-as-Tool” paradigm , introducing a novel strategy that treats MLLMs as specialized visual processing tools to efficiently decompose and solve complex visual-textual tasks. • Extensive experimental validation , including online A/B testing and simulation-based ablation studies, confirming MindFlow’s effectiveness, robustness, and scalability in realistic e-commerce scenarios. 2 Related Work Language agents are an emerging class of AI systems that integrate LLMs with external tools, memory modules, and environment interfaces to enable autonomous reasoning and decision-making ( Xi et al. 2023 ; Weng 2023 ; Wang et al. 2024 ) . Unlike static prompting, these agents operate in a closed-loop fashion: they formulate actions, receive feedback, and iteratively update internal state. Influential methods such as ReAct ( Yao et al. 2023 ) combine step-by-step reasoning with tool use, allowing agents to interleave thoughts and actions for enhanced problem-solving. Reflexion ( Shinn et al. 2023 ) further introduces self-evaluation and retrospective learning, enabling iterative strategy refinement. Building upon these foundations, recent systems like CAMEL ( Li et al. 2023 ) support multi-agent collaboration, whereas SWE-agent ( Yang et al. 2024 ) specializes in domain-specific workflows. Meanwhile, autonomous agents such as Manus and BabyAGI ( Nakajima 2023 ) investigate open-ended task decomposition and planning in dynamic environments. In the domain of e-commerce customer service, LLM agents are increasingly explored to blend automated assistance with human-in-the-loop workflows. While public documentation on commercial deployments remains limited, systems such as Crescendo, Sierra AI, and Alibaba’s 1688 platform have reportedly integrated AI agents for recommendation, dialogue, and post-sale support. Complementing these applications, recent research proposes instruction-tuned and domain-adapted models ( Herold et al. 2024 ; Peng et al. 2024 ; Zhang et al. 2024 ) , leveraging structured data and multitask learning to tackle challenges like product understanding and customer intent classification. These developments underscore the growing emergence of LLM agents in e-commerce applications. 3 MindFlow Figure 1: MindFlow Architecture MindFlow, implemented using OpenAI Agent SDK, functions as an agentic agent capable of invoking external tools encapsulated in the Model Context Protocol (MCP) format. Figure 1 presents the overall architecture of MindFlow, which processes buyer queries through a tightly integrated loop of decision-making, memory, and action modules to generate reliable and context-aware responses. Its capability to handle multimodal scenarios is further enhanced by the integration of the “MLLM-as-Tool” strategy. 3.1 Memory Module The memory module is introduced to enhance contextual understanding and knowledge retention in complex e-commerce tasks. It consists of two complementary components: working memory and long-term memory. The working memory captures the historical interaction between the buyer and the customer representative. It is accessed by the decision-making module during the Propose stage to help infer buyer intent, particularly when queries are implicit or incomplete. In contrast, the long-term memory stores domain-specific knowledge, including platform policies, store-level promotions, product details, and buyer-specific data such as order and logistics information. This component is retrieved through internal actions in the action module and is continuously updated as new information emerges during interaction. Both memory types are essential. Without long-term memory, the agent lacks the necessary background to perform any meaningful reasoning; without working memory, it cannot accurately interpret buyer intent, rendering tool invocation ineffective. Together, they empower the agent to reason with context, act with precision, and adapt dynamically across diverse e-commerce scenarios. 3.2 Decision-Making Module The decision-making module is responsible for generating optimal action plans based on the current system state and memory. To this end, we adopt a “Propose-Evaluate-Select” framework, where the agent first generates multiple candidate plans, evaluates them based on their ability to fulfill the buyer intent, and deterministically selects the best one for execution. Candidate plans may consist of sequences of tool invocations, single-tool calls, or direct responses without tool use. This framework encourages the agent to consider diverse possibilities and avoid shortsighted decisions limited to immediate next steps. By modeling plan selection as a discrete classification task, it further enables reliable confidence estimation, capitalizing on LLMs’ intrinsic calibration properties in single-token output tasks Achiam et al. 2023 ; Qun et al. 2024 . The resulting confidence scores reflect the reliability of each option, supporting more interpretable and trustworthy decision-making. This structured approach facilitates adaptability to diverse scenarios, transparent evaluation through explicit scoring, and robustness in action selection, significantly improving agent performance in complex and dynamic e-commerce interactions. 3.3 Action Module The action module defines all executable operations in MindFlow, including both external and internal actions. This design enables precise and flexible behavior in dynamic e-commerce environments. External actions refer to the invocation of predefined tools that allow the agent to interact with environment and retrieve task-relevant information. Internal actions, in contrast, involve intra-agent operations such as memory retrieval, internal reasoning steps, and status updates. MindFlow incorporates a specialized mechanism inspired by the Agent-Computer Interface (ACI) Yang et al. 2024 to simplify input complexity and enable

Read the complete text