Pragmatic Theories Enhance Understanding of Implied Meanings in LLMs

Takuma Sato, Seiya Kawano, Koichiro Yoshino · 2025

Read Pragmatic Theories Enhance Understanding of Implied Meanings in LLMs by Takuma Sato, Seiya Kawano, Koichiro Yoshino (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-26253.txt.

Introduction

Takuma Sato, Seiya Kawano, and Koichiro Yoshino show that teaching a language model a little pragmatics helps it read implied meanings. Pragmatics is the study of how context shapes meaning beyond literal words, and grasping implication is central to human communication. Their approach places an overview of pragmatic theories, such as Gricean pragmatics and Relevance Theory, into the prompt as in context guidance.

The model then reasons step by step toward an interpretation, guided by the theory. Compared with a baseline that prompts intermediate reasoning without theory, their method scored up to nine point six percent higher on pragmatic reasoning tasks in their experiments. Even naming the theories without detail gave larger models a small gain of about one to three percent.

Catalog record

CREATORTakuma Sato, Seiya Kawano, Koichiro Yoshino
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE47 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-26253

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Pragmatic Theories Enhance Understanding of Implied Meanings in LLMs Takuma Sato, Seiya Kawano, Koichiro Yoshino Abstract † † footnotetext: This is the preprint version of the paper accepted to the main conference of IJCNLP-AACL 2025. The ability to accurately interpret implied meanings plays a crucial role in human communication and language use, and language models are also expected to possess this capability. This study demonstrates that providing language models with pragmatic theories as prompts is an effective in-context learning approach for tasks to understand implied meanings. Specifically, we propose an approach in which an overview of pragmatic theories, such as Gricean pragmatics and Relevance Theory, is presented as a prompt to the language model, guiding it through a step-by-step reasoning process to derive a final interpretation. Experimental results showed that, compared to the baseline, which prompts intermediate reasoning without presenting pragmatic theories (0-shot Chain-of-Thought), our methods enabled language models to achieve up to 9.6% higher scores on pragmatic reasoning tasks. Furthermore, we show that even without explaining the details of pragmatic theories, merely mentioning their names in the prompt leads to a certain performance improvement (around 1-3%) in larger models compared to the baseline. 1 Introduction Language often contains implicit meanings, unstated intentions, and context-dependent interpretations, collectively referred to as implied meanings . The ability to correctly understand and interpret implied meanings play a crucial role in human communication Grice 1989 ; Levinson 1983 ; Carston 2002 . In order to correctly interpret the implied meanings, people use background knowledge of communication, such as dialogue context, common sense, and cultural background, and achieve appropriate communication in their daily activities. The ability to understand implied meanings is important not only for humans but also for AI systems or robots based on them. For example, when a human’s intention is ambiguous, inferring their intentions and taking actions proactively while considering the situation and common sense is necessary Tanaka et al. 2024 . The ability to interpret implied meaning, as well as understanding the nuances and intentions of language, is essential for AI and robots to be needed in human society. In recent years, following the success of Large Language Models (LLMs), research has been conducted on employing them to develop systems that have abilities to interpret language and situation Ahn et al. 2022 ; Ma et al. 2025 . Numerous benchmark tasks and datasets have been proposed to assess these capabilities Jeretic et al. 2020 ; Zheng et al. 2021 ; Takayama et al. 2021 ; Hu et al. 2023 ; Li et al. 2024a ; Sravanthi et al. 2024 ; Yerukola et al. 2024 ; Yue et al. 2024 , facilitating comparisons between current language models and human performance, as well as analyses of the challenges these models face.Numerous benchmark tasks and datasets have been proposed to assess these capabilities Jeretic et al. 2020 ; Zheng et al. 2021 ; Takayama et al. 2021 ; Hu et al. 2023 ; Li et al. 2024a ; Sravanthi et al. 2024 ; Yerukola et al. 2024 ; Yue et al. 2024 , facilitating comparisons between current language models and human performance, as well as analyses of the challenges these models face. Two major approaches have been explored to enhance the understanding of implied meanings in LLMs: post-training involving parameter updates Wu et al. 2024 ; Sravanthi et al. 2024 , and in-context learning (ICL), which draws out specific capabilities of LLMs through carefully designed prompts Yerukola et al. 2024 ; Ruis et al. 2024 ; Kim et al. 2023 . ICL is particularly important, as it enables the full utilization of the model’s abilities without additional training costs, and has been addressed with extensive approaches regarding its generalization capabilities. In the context of understanding implied meanings, there is a growing need to explore not only instance-specific methods but also more comprehensive, instance-agnostic methods. In this study, we propose an in-context learning method that enhances model performance on tasks to understand implied meanings in a zero-shot setting without relying on prompts specific to particular problem formats or providing top-down hints from correct answers. Implied meanings are handled in the field of linguistics known as pragmatics , where various theories have been proposed regarding their properties and the mechanisms of their interpretation. The proposed method inserts summaries of theoretical frameworks from linguistic pragmatics into the model’s prompt and instructs the model to generate intermediate reasoning processes by following those theories. It is known that existing LLMs have a variety of knowledge in their parameters; however, making them recall and operate knowledge appropriately and for the task is still challenging. By giving the rough outline of pragmatic theories as a bootstrap, we expected that these models could recall the related knowledge to be used and manipulate them for solving such tasks to understand the implied meanings. Experimental results demonstrated that the proposed method consistently improved model performance on tasks to understand implied meanings without providing instance-dependent information, achieving an accuracy improvement of up to 0.096 in the experimental tasks. We tested the proposed method in both commercial-closed and open models and found that the proposed method contribute to a wide range of models. Notably, GPT-4o achieved higher scores than human scores by applying the proposed method. Additionally, slight score improvements were observed in many models even when the prompt did not include an overview of pragmatic theories but only mentioned their names while encouraging reasoning by following them. These investigations not only have engineering utility and can be expected to apply to upstream tasks such as dialogue, but also lead to clarifying the nature of the task of pragmatic understanding itself and what kind of thought processes are effective for executing such tasks. Our contributions can be summarized as follows: • We proposed a simple in-context learning method that incorporates summaries of pragmatic theories (namely Gricean and Relevance Theories) into prompts and showed that this improves LLM performance on implied meaning understanding tasks without preparing task-dependent prompts. • We showed that even without explaining the theories in detail, simply referencing theory names in zero-shot Chain-of-Thought prompting leads to performance improvements over baseline methods. • Through detailed quantitative and qualitative analyses, we demonstrated that our method is particularly effective for tasks involving utterances that flout Grice’s maxims and for interpreting irony. 2 Related Works 2.1 Pragmatic Reasoning by Language Models Previous research has demonstrated that methods such as post-training, Chain-of-Thought (CoT) reasoning, and few-shot learning effectively enhance the pragmatic inference capabilities of language models. As post-training approaches using pragmatic reasoning datasets, policy optimization methods such as Direct Preference Optimization (DPO) are more effective than supervised fine-tuning Wu et al. 2024 . It has also been confirmed that instruction-tuned models achieve higher scores in pragmatic reasoning tasks compared to their corresponding base models Ruis et al. 2024 ; Sravanthi et al. 2024 . These methods involve updating the model’s parameters. For in-context learning methods that do not involve updating the model’s parameters, previous research has shown that CoT prompting, where guidances for correct interpretation are included in the prompt Yerukola et al. 2024 , and few-shot prompting, where the prompts provide examples of problems and correct answers similar to the target task Ruis et al. 2024 , are

Read the complete text