Intuitionistic Fuzzy Sets for Large Language Model Data Annotation: A Novel Approach to Side-by-Side Preference Labeling

Yimin Du · 2025

Read Intuitionistic Fuzzy Sets for Large Language Model Data Annotation: A Novel Approach to Side-by-Side Preference Labeling by Yimin Du (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-24199.txt.

Introduction

Yimin Du reworks how human judges label AI outputs in a 2025 paper. Side-by-side picks train and grade large language models, but judges often feel unsure or split. Plain yes-no labels and fixed scales flatten that doubt and waste signal.

Du uses intuitionistic fuzzy sets, which track support, pushback, and hesitation at once. The frame adds a labeling protocol, ways to merge split judges, and quality scores for label sets. Tests across data sets showed steadier labels, less judge fatigue, and faster work. Models trained on the richer labels also won more head-to-head rounds. The frame gives doubt a formal place in label work.

Catalog record

CREATORYimin Du
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE27 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-24199

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Intuitionistic Fuzzy Sets for Large Language Model Data Annotation: A Novel Approach to Side-by-Side Preference Labeling Yimin Du Abstract The quality of human preference data is crucial for training and evaluating large language models (LLMs), particularly in reinforcement learning from human feedback (RLHF) and direct preference optimization (DPO) scenarios. Traditional side-by-side (SBS) annotation approaches often struggle with inherent uncertainty, annotator disagreement, and the complexity of preference judgments. This paper introduces a novel framework based on intuitionistic fuzzy sets (IFS) for modeling and aggregating human preferences in LLM data annotation tasks. Our approach captures not only the degree of preference but also the uncertainty and hesitation inherent in human judgment through membership, non-membership, and hesitation degrees. We propose an IFS-based annotation protocol that enables more nuanced preference modeling, develops aggregation methods for handling annotator disagreement, and introduces quality metrics for preference data assessment. Experimental validation on multiple datasets demonstrates that our IFS-based approach significantly improves annotation consistency, reduces annotator fatigue, and produces higher-quality preference data compared to traditional binary and Likert-scale methods. The resulting preference datasets lead to improved model performance in downstream tasks, with 12.3% improvement in win-rate against baseline models and 15.7% reduction in annotation time. Our framework provides a principled approach to handling uncertainty in human preference annotation and offers practical benefits for large-scale LLM training. I Introduction The rapid advancement of large language models (LLMs) has fundamentally transformed natural language processing, with models like GPT-4, Claude, and Gemini demonstrating remarkable capabilities across diverse tasks. However, the effectiveness of these models heavily depends on high-quality training data, particularly human preference data used in reinforcement learning from human feedback (RLHF) [ 1 ] and direct preference optimization (DPO) [ 2 ] . The quality of preference annotations directly impacts model alignment, safety, and overall performance. Traditional approaches to preference annotation typically employ binary choices or Likert scales in side-by-side (SBS) evaluation scenarios, where annotators compare two or more model outputs and indicate their preference. However, these methods face several fundamental challenges: Inherent Uncertainty: Human preferences often involve uncertainty, especially when comparing outputs of similar quality or when dealing with subjective criteria. Traditional binary annotation fails to capture this uncertainty, forcing annotators to make definitive choices even when they are genuinely uncertain. Annotator Disagreement: Different annotators may have varying preferences based on their expertise, cultural background, or interpretation of evaluation criteria. Current aggregation methods often oversimplify this disagreement through majority voting or averaging, potentially losing valuable information about the diversity of human judgment. Complexity of Preference Judgments: Real-world preference decisions involve multiple criteria (fluency, accuracy, helpfulness, safety) that may conflict with each other. Traditional annotation schemes struggle to capture the multifaceted nature of these judgments. Annotation Fatigue: The cognitive burden of making definitive preference decisions, especially for subtle differences, can lead to annotator fatigue and reduced annotation quality over time. To address these challenges, we propose a novel framework based on intuitionistic fuzzy sets (IFS) [ 3 ] for modeling human preferences in LLM data annotation. IFS extends classical fuzzy sets by introducing three fundamental components: membership degree (support for a preference), non-membership degree (opposition to a preference), and hesitation degree (uncertainty or indecision). This mathematical framework naturally captures the complexity and uncertainty inherent in human preference judgments. Our main contributions are: IFS-Based Annotation Framework: We develop a comprehensive annotation protocol that allows annotators to express preferences using membership, non-membership, and hesitation degrees, providing a more nuanced representation of human judgment. Aggregation Methods for IFS Preferences: We propose novel aggregation techniques specifically designed for combining IFS-based preference annotations from multiple annotators, preserving uncertainty information while producing reliable consensus judgments. Quality Assessment Metrics: We introduce IFS-specific quality metrics for evaluating annotation consistency, uncertainty levels, and overall data quality, enabling better quality control in large-scale annotation projects. Empirical Validation: We conduct extensive experiments on multiple datasets and tasks, demonstrating that our IFS-based approach improves annotation quality, reduces annotator fatigue, and leads to better downstream model performance. Practical Implementation: We provide a complete implementation framework including annotation interfaces, aggregation algorithms, and integration methods for existing LLM training pipelines. The remainder of this paper is organized as follows: Section II reviews related work in preference annotation and fuzzy set applications. Section III introduces the theoretical foundation of our IFS-based approach. Section IV details the annotation framework and aggregation methods. Section V presents experimental validation and results. Section VI discusses practical implications and limitations. Section VII concludes with future directions. II Related Work II-A Human Preference Annotation for LLMs Human preference annotation has become a cornerstone of modern LLM training, particularly following the success of InstructGPT [ 1 ] . The standard approach involves presenting annotators with pairs of model outputs and asking them to indicate which response is better according to specific criteria such as helpfulness, harmlessness, and honesty. Recent work has explored various aspects of preference annotation quality. Kreutzer et al. [ 4 ] investigated the reliability of human judgments in machine translation evaluation, highlighting significant inter-annotator disagreement. Ethayarajh et al. [ 5 ] analyzed the consistency of human preferences in text generation tasks, finding that annotator agreement varies significantly across different types of tasks and criteria. Several studies have attempted to improve annotation quality through better interface design [ 6 ] , clearer guidelines [ 7 ] , and annotator training [ 8 ] . However, these approaches primarily focus on improving the annotation process rather than fundamentally addressing the uncertainty inherent in human judgment. II-B Uncertainty in Human Annotation The challenge of uncertainty in human annotation has been recognized across various NLP tasks. Plank et al. [ 9 ] demonstrated that annotator disagreement often contains valuable signal rather than noise, suggesting that preserving uncertainty information can be beneficial for model training. In the context of preference annotation, recent work has begun to explore alternatives to binary choices. Dubois et al. [ 10 ] introduced multi-way comparisons and tie options, while Bai et al. [ 11 ] explored constitutional AI approaches that incorporate uncertainty in preference modeling. However, existing approaches to handling uncertainty remain limited, typically involving simple extensions like ”tie” options or confidence scores, without providing a principled mathematical framework for uncertainty representation. II-C Fuzzy Sets in Natural Language Processing Fuzzy set theory, introduced by Zadeh [ 12 ] , has been applied to various NLP tasks where uncertainty and gradual membership are important. Applications include sentiment analysis [ 13 ] , text classification [ 14 ] , and information retrieval [ 15 ] . Intuitionistic fuzzy sets, proposed by Atanassov [ 3 ] , extend classical fuzzy sets by explicitly modeling both membership and non-membership degrees, along with a hesitation degree representing uncertainty. IFS has been successfully applied to decision-making problems [ 16 ] , multi-criteria evaluation [ 17 ] , and group decision support systems [ 18 ] . Despite the natural fit between IFS and preference modeling, there has been limited exploration of IFS applications in LLM annotation tasks. Our work fills this gap by developing a comprehensive IFS-based framework specifically designed for preference annotation in LLM training. II-D Aggregation Methods for Fuzzy Preferences The aggregation of fuzzy preferences has been extensively studied in decision theory and multi-criteria decision making. Xu [ 16 ] proposed various aggregation operators

Read the complete text