EAVIT: Efficient and Accurate Human Value Identification from Text data via LLMs

Wenhao Zhu, Yuhang Xie, Guojie Song, Xin Zhang · 2025

Read EAVIT: Efficient and Accurate Human Value Identification from Text data via LLMs by Wenhao Zhu, Yuhang Xie, Guojie Song, Xin Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-12792.txt.

Introduction

Wenhao Zhu, Yuhang Xie, Guojie Song, and Xin Zhang present a framework in two thousand twenty-five for spotting human values in text. Language models now beat older tools at reading values from writing, but feeding them long texts is costly and their accuracy slips on long inputs.

Their system, EAVIT, pairs a small local model with a large online one. The local detector makes first-pass value estimates, which are packed into short prompts for the online model to judge. Special training data and sampling keep those prompts brief. Input tokens fell by up to five-sixths versus asking the online model directly. Accuracy still beat traditional methods and other model-based strategies. The design trades a little local compute for large savings.

Catalog record

CREATORWenhao Zhu, Yuhang Xie, Guojie Song, Xin Zhang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE63 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-12792

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

EAVIT: Efficient and Accurate Human Value Identification from Text data via LLMs Wenhao Zhu, Yuhang Xie, Guojie Song, Xin Zhang Abstract The rapid evolution of large language models (LLMs) has revolutionized various fields, including the identification and discovery of human values within text data. While traditional NLP models, such as BERT, have been employed for this task, their ability to represent textual data is significantly outperformed by emerging LLMs like GPTs. However, the performance of online LLMs often degrades when handling long contexts required for value identification, which also incurs substantial computational costs. To address these challenges, we propose EAVIT, an efficient and accurate framework for human value identification that combines the strengths of both locally fine-tunable and online black-box LLMs. Our framework employs a value detector—a small, local language model—to generate initial value estimations. These estimations are then used to construct concise input prompts for online LLMs, enabling accurate final value identification. To train the value detector, we introduce explanation-based training and data generation techniques specifically tailored for value identification, alongside sampling strategies to optimize the brevity of LLM input prompts. Our approach effectively reduces the number of input tokens by up to 1/6 compared to directly querying online LLMs, while consistently outperforming traditional NLP methods and other LLM-based strategies. 1 Introduction The recent advent of Large Language Models (LLMs), including the Generative Pre-trained Transformer (GPT) models Brown et al. (2020) ; OpenAI (2023) and Llama Touvron et al. (2023a) , has marked a pivotal advancement in natural language processing (NLP). One notable application of LLMs is the discovery and identification of human values from text data, such as arguments Kiesel et al. (2022) ; Kiesel et al. (2023) . This task holds significant potential across various domains, supporting value alignment for LLMs Yao et al. (2023) , value-based argument models Atkinson and Bench-Capon (2021) , and computational psychological studies Alshomary et al. (2022) . Before the era of LLMs, identifying human values from text data in computational linguistics was typically framed as a multi-label classification problem and addressed using machine learning and NLP models like BERT Devlin et al. (2019) . However, these models are now being outperformed by modern LLMs in terms of their general text comprehension capabilities. Since most LLMs are accessible only through black-box APIs and fine-tuning them is computationally and economically inefficient, the standard approach for utilizing LLMs in value identification involves constructing tailored prompts for direct queries. For LLMs to effectively identify values from text data, they must first learn the value system definition , akin to how humans do. This definition is typically presented as a long context—for instance, the basic Schwartz values definition spans approximately 2.5k tokens Schwartz (2012) . Including such lengthy definitions in the input prompt is not only costly but has also been shown to degrade performance in context-heavy tasks Liu et al. (2023) , a finding corroborated by our experiments. Figure 1: An illustration of (a) human value system and (b) the proposed EAVIT method compared with directly using LLMs. Given these challenges, to better leverage the capabilities of LLMs, we propose EAVIT, a framework for E fficient and A ccurate H uman V alue I dentification from Text data. EAVIT begins with a value detector (a tunable local language model) that generates initial value estimations. These estimations are then used to construct a concise input prompt for the LLM, enabling accurate final value identification. The value detector identifies values that are most certainly related and those that are most certainly unrelated, leaving only values with uncertain relevance to be resolved by the LLM. This approach effectively reduces the number of values requiring explicit definition in the input context, thereby shortening the overall context length. To enable the value detector to learn the cognitive logic underlying value identification, we employ an explanation-based fine-tuning method, training the model to reflect on the definitions of values throughout the learning process. To address issues such as insufficient training data and imbalanced class distributions, we draw inspiration from methods like Self-Instruct Wang et al. (2022) and utilize diverse data generation techniques via LLMs to create high-quality training datasets. Additionally, to ensure an optimal candidate set, we perform multiple sampling rounds to identify the most relevant candidate values. The proposed EAVIT framework not only achieves state-of-the-art performance but also significantly reduces the token cost for inference—down to nearly \frac{1}{6} of the tokens required when directly using LLMs. This makes it a promising and cost-effective solution for large-scale value identification tasks. Our contributions can be summarized as follows: • We introduce EAVIT, a novel framework for identifying human values from text data. This methodology efficiently leverages the power of LLMs, offering accurate and cost-effective value identification results. • We employ a diverse set of training strategies for the value detector (local LM in EAVIT), including explanation-based fine-tuning and data generation techniques. Additionally, we develop sampling and prompt-generation methods to create concise and effective input prompts for LLMs. • Our approach achieves state-of-the-art performance compared to traditional NLP methods and direct LLM-based strategies. Furthermore, it significantly reduces inference costs to as low as \frac{1}{6} of the tokens required by conventional LLM approaches, making it highly scalable for tasks such as LLM alignment and psychological analysis. 2 Related Works Human Value Theories A detailed review of human value theories in psychology can be found in Appendix. In our paper, following existing works Kiesel et al. (2023) in computational linguistics, we adopt Schwartz’s Theory of Basic Values Schwartz (2012) as the basic value system for value identification, which has been applied in multiple fields including economics Ng et al. (2005) and LLMs Miotto et al. (2022) ; Fischer et al. (2023) . Meanwhile, it should be noted that our method is applicable to any completely defined value system (such as the extended Schwartz value system or those with values set for language models). Figure 2: An illustration of the Schwartz value systems. Value Identification from Text Data. In NLP, the identification of human values from text data can be perceived as a multi-label classification or regression task described by complex task definition. Recent key related works include Qiu et al. (2022) ; Kiesel et al. (2022) ; Kiesel et al. (2023) ; Ren et al. (2024) . In Qiu et al. (2022) , simple social scenario descriptions from Forbes et al. (2020) were selected and annotated using Schwartz’s value taxonomy, establishing the ValueNet dataset for value modeling in language models. Kiesel et al. (2022) was the first to systematically establish the task of identifying hidden human values from argument data and built a dataset, Webis-ArgValues-22, of 5k size derived from social network data and annotated with Schwartz values by humans. Kiesel et al. (2023) extended the work of Kiesel et al. (2022) with dataset Touché23-ValueEval, expanding the dataset size to 9k and held a public competition at the ACL2023 workshop. Yao et al. (2023) also proposes FULCRA dataset (currently unavailable) that labels LLM outputs to Schwartz human values. Our experiments will use these public, human-annotated datasets as the basis for training and validation. Touché23-ValueEval will be the main dataset. 3 Human Value Identification - Task and Basic Methods 3.1 Task Definition We first introduce the formal

Read the complete text