AgreeMate: Teaching LLMs to Haggle
Ainesh Chatterjee Samuel Miller Nithin Parepally [email protected] [email protected] [email protected] Department of Computer Science, University of Maryland, College Park, USA · 2024
Read AgreeMate: Teaching LLMs to Haggle by Ainesh Chatterjee Samuel Miller Nithin Parepally [email protected] [email protected] [email protected] Department of Computer Science, University of Maryland, College Park, USA (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-18690.txt.
Introduction
Ainesh Chatterjee, Samuel Miller, and Nithin Parepally teach language models to bargain in a 2024 paper. Their framework, AgreeMate, trains models to haggle over prices in plain words. Two agents, a buyer and a seller, trade offers through a modular setup where thinking and talking are split into parts.
Prompt design, fine tuning, and step by step reasoning prompts each lifted bargaining skill, judged by new measures the authors define. Probes of the model's attention showed it tracking the meaning links between words during talks, such as price terms and product traits. The work tests how far language models can go as negotiating agents.
Catalog record
| CREATOR | Ainesh Chatterjee Samuel Miller Nithin Parepally [email protected] [email protected] [email protected] Department of Computer Science, University of Maryland, College Park, USA |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-18690
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
AgreeMate: Teaching LLMs to Haggle Ainesh Chatterjee Samuel Miller Nithin Parepally [email protected] [email protected] [email protected] Department of Computer Science, University of Maryland, College Park, USA Abstract We introduce AgreeMate , a framework for training Large Language Models (LLMs) to perform strategic price negotiations through natural language. We apply recent advances to a negotiation setting where two agents (i.e. buyer or seller) use natural language to bargain on goods using coarse actions. Specifically, we present the performance of Large Language Models when used as agents within a decoupled (modular) bargaining architecture. We demonstrate that using prompt engineering, fine-tuning, and chain-of-thought prompting enhances model performance, as defined by novel metrics. We use attention probing to show model attention to semantic relationships between tokens during negotiations. 1 Introduction The emergence of Large Language Models (LLMs) has transformed our understanding of machine intelligence, demonstrating remarkable capabilities in language comprehension and complex social interactions. While recent works have explored LLMs’ abilities in dialogue and reasoning tasks, their potential in strategic communication scenarios remains underexplored. We present AgreeMate, a comprehensive testing framework and negotiation system that explores the capabilities of LLMs in strategic price bargaining. Our approach combines role-specialized fine-tuning with systematic model comparison across scales and architectures, offering insights into how different LLM variants perform in complex negotiation scenarios. Strategic negotiation represents a particularly challenging domain for artificial intelligence, requiring agents to simultaneously master natural language generation, maintain consistent goal-directed behavior, and adapt tactics based on counterpart responses. Previous approaches to automated negotiation often relied on complex architectures with separate modules for strategy planning and language generation ( He et al., 2018 ) . However, the emergence of instruction-tuned LLMs offers an opportunity to revisit this problem with potentially simpler, more elegant solutions that leverage these models’ inherent reasoning and natural dialogue capabilities. Our work makes several key contributions: 1. We introduce a systematic framework for evaluating LLM negotiation capabilities across different model scales, architectures, and training approaches. 2. We develop and analyze role-specialized negotiation agents (buyer, seller, generalist) through targeted fine-tuning, demonstrating the effectiveness of role-specific optimization. 3. We present a detailed comparative analysis between base LLaMA models (3B to 70B parameters), revealing key insights about the model scale and training impacts on negotiation performance ( Touvron et al., 2023 ) . 4. We provide comprehensive evaluation metrics for negotiation success, including derived and novel measurements such as fairness, bias, relative efficiency and probing ratio. 5. We conduct attention head probing to gain deeper insights into the internal workings of LLMs during strategic negotiation, shedding light on how specific attention mechanisms contribute to bargaining behavior. This project was conducted as part of the CMSC723: Graduate Natural Language Processing (Fall 2024) course at the University of Maryland, College Park. The content and findings reflect our independent research and do not imply any endorsement or ownership by the university. 2 Motivation The capabilities of modern large language models (LLMs) have advanced significantly, with models like GPT-4 demonstrating persuasive abilities that reportedly exceed internal safety benchmarks ( Seetharaman, 2024 ) . These models can craft highly effective arguments using psycholinguistic principles ( Breum et al., 2023 ) , raising important questions about their application in structured domains such as price negotiation. Negotiation serves as an ideal testbed for LLMs, requiring a delicate balance between maximizing utility and fostering cooperation. Effective negotiation relies on implicit strategies—revealing one’s true intentions or limits typically weakens their position. This inherent opacity aligns well with the “black box” reasoning of LLMs, which can emulate human-like strategic decision-making without needing explicit disclosures. Previous automated negotiation systems ( He et al., 2018 ) relied on modular architectures to separate strategic planning from natural language generation. While effective for their time, these approaches reflected the limitations of pre-transformer models, which struggled with multi-step reasoning. In contrast, instruction-tuned LLMs like LLaMA ( Touvron et al., 2023 ) offer the potential to unify these tasks, discovering nuanced negotiation strategies through exposure to domain-specific dynamics rather than manual engineering. Developing effective automated negotiation agents has practical implications. Such systems could assist in online marketplaces, enabling efficient price discovery and ensuring fair transactions by either providing strategic advice or directly participating in negotiations. These agents can address the growing need for scalable and impartial negotiation mechanisms in increasingly digital economies. This work examines how LLM scale, post-training procedures, and role-specific fine-tuning impact negotiation performance. By comparing models ranging from 3B to 70B parameters, we aim to identify the minimal model capabilities required for effective negotiation. This research also explores how these systems can be deployed in resource-constrained settings without compromising performance, offering insights into the trade-offs between model complexity and strategic efficacy. 3 Architecture Our work takes inspiration from the "Decoupling Strategy and Generation in Negotiation Dialogues" ( He et al., 2018 ) paper in which buyer and seller dialogue agents produce a distribution over a responding utterance given a dialogue history (a sequence of utterances). The modular framework determining an agent’s responding utterance is characterized as follows. A parser module maps input utterances into a coarse dialogue act given a dialogue history and previous dialogue acts. Acts are selected from following: greet, disagree, agree, insist, inquire, inform, counter, intro, vague-price, intro, and propose. Here, course dialogue acts do not capture the entire semantics of a sentence, which enables a rule-based approach to parsing. The parser works by detecting prices and objects referred to as “entities” and matches keyword patterns. Then, a manager module predicts a dialogue act given information about the scenario and past dialogue acts. The paper uses three approaches to train the manager which include supervised learning, where a sequence to sequence model with attention is used to learn transition probabilities of dialogue acts, reinforcement learning where training consists of optimizing for a reward function, and finally a hybrid approach where a learned manager is combined with hand-coded rules. For the reinforcement learning approach, three different reward functions are used. These are utility, fairness, and length. Finally, a generator module uses retrieval based generation, where a database of candidate utterances is searched for the best match based on the act predicted by the manager and the history, to output a responding utterance. Utterances in the database are in a template-based format where placeholders are replaced with contextually relevant information during test time. In our project, we adopt a similar framework to make the task suitable for an LLM. In particular, our framework eliminates the parser component and the LLM serves as both the manager and the generator. Given only a history of utterances (so no dialogue acts), scenario information, and the number of turns left in the negotiation (we enforce a limit), a buyer or seller agent selects an act from { intro, init-price, offer, counter-price, insist, agree, disagree, accept, inform, inquire, and unknown }. Then, based on the selected action, a contextually appropriate utterance aligning with the action is produced. These outputs are formatted into what we call action-utterance pairs. Our “manager” component does not vary with approaches like the paper which uses a supervised learning approach to mimic average human behavior and a reinforcement learning approach to directly optimize for a particular goal. Our framework provides the same set of instructions to each agent comprised