From Classification to Ranking: Enhancing LLM Reasoning Capabilities for MBTI Personality Detection

Yuan Cao, Feixiang Liu, Xinyue Wang, Yihan Zhu, Hui Xu, Zheng Wang, Qiang Qiu · 2026

Read From Classification to Ranking: Enhancing LLM Reasoning Capabilities for MBTI Personality Detection by Yuan Cao, Feixiang Liu, Xinyue Wang, Yihan Zhu, Hui Xu, Zheng Wang, Qiang Qiu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-18582.txt.

Introduction

Yuan Cao and six colleagues recast a trait-guess task as a rank task to get more from language models. The task reads a person's posts and guesses their type on a famed four-scale quiz of traits. Past tools had models pull key text from posts and then train a sorter to pick one type. That pick is coarse: a near miss counts the same as a wild one.

A rank list is finer. It can put the true type high even when the top pick is off. The team train and grade models to rank types, not just to name one. A finer grade should show small gains that a plain pick would hide. It may also give users a range of fits, not a flat tag. Trait tools are used in hires and ads, so fine grades count.

Catalog record

CREATORYuan Cao, Feixiang Liu, Xinyue Wang, Yihan Zhu, Hui Xu, Zheng Wang, Qiang Qiu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-18582

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

From Classification to Ranking: Enhancing LLM Reasoning Capabilities for MBTI Personality Detection Yuan Cao, Feixiang Liu, Xinyue Wang, Yihan Zhu, Hui Xu, Zheng Wang, Qiang Qiu Abstract Personality detection aims to measure an individual’s corresponding personality traits through their social media posts. The advancements in Large Language Models (LLMs) offer novel perspectives for personality detection tasks. Existing approaches enhance personality trait analysis by leveraging LLMs to extract semantic information from textual posts as prompts, followed by training classifiers for categorization. However, accurately classifying personality traits remains challenging due to the inherent complexity of human personality and subtle inter-trait distinctions. Moreover, prompt-based methods often exhibit excessive dependency on expert-crafted knowledge without autonomous pattern-learning capacity. To address these limitations, we view personality detection as a ranking task rather than a classification and propose a corresponding reinforcement learning training paradigm. First, we employ supervised fine-tuning (SFT) to establish personality trait ranking capabilities while enforcing standardized output formats, creating a robust initialization. Subsequently, we introduce Group Relative Policy Optimization (GRPO) with a specialized ranking-based reward function. Unlike verification tasks with definitive solutions, personality assessment involves subjective interpretations and blurred boundaries between trait categories. Our reward function explicitly addresses this challenge by training LLMs to learn optimal answer rankings. Comprehensive experiments have demonstrated that our method achieves state-of-the-art performance across multiple personality detection benchmarks. Introduction Language-based personality detection is crucial for AI applications such as mental health screening and conversational agents, utilizing established psychological frameworks like Myers-Briggs Type Indicator (MBTI) (Myers 1962; Myers and McCaulley 1985). As illustrated in Figure 1, MBTI characterizes personality along four interdependent dimensions: Extraversion (E) vs. Introversion (I), Sensing (S) vs. iNtuition (N), Thinking (T) vs. Feeling (F), and Judging (J) vs. Perceiving (P). Crucially, these dimensions exhibit dynamic interactions governed by hierarchical cognitive functions that collectively shape behavioral expressions. For instance, in MBTI theory, an INTJ’s dominant function is introverted intuition (Ni), while ENTJ’s dominant function is extraverted thinking (Te), leading to different cognitive profiles even if only one letter differs. Prevailing approaches (Yang et al. 2021b; Zhu et al. 2024) fundamentally limit their efficacy by treating MBTI prediction as four independent binary classification tasks (”E/I”, ”S/N”, ”T/F”, ”J/P”), resulting in two critical shortcomings: dimensional decoupling that ignores psychometrically established interactions, static labeling that fails to capture complete personality expressions which neglects the position on the scale (Stajner and Yenikent 2021). While traditional models struggle with this interdependence, the recent success of Large Language Models (LLMs) offers new opportunities to approach personality modeling in a more holistic and reasoning-driven manner. Currently, LLMs exhibit remarkable reasoning capabilities across diverse NLP tasks (Brown et al. 2020; Wei et al. 2022), but their application to personality assessment reveals fundamental limitations. Prompt-based methods such as PsyCoT (Yang et al. 2023b) mimic the way individuals complete psychological questionnaires through multi-turn dialogues. However, the limitations of prompt-based approaches are their over-reliance on expert-defined prompts and large-scale parameter models (Le et al. 2024; Islam et al. 2025), which makes them impractical for real-time applications due to high inference latency and computational cost. This gap necessitates novel frameworks capable of autonomously learning discriminative patterns from behavioral data. Figure 1: A framework for LLM-based MBTI personality detection. The framework leverages a single user’s social media posts as input to corresponding personality traits along four binary dimensions of Introversion vs. Extroversion, Sensing vs. iNtuition, Think vs. Feeling, and Perception vs. Judging. To resolve these limitations, we pioneered a paradigm shift from categorical classification to preference ranking, a transformation aligned with psychometric realities where personality manifests itself through relative trait preferences rather than absolute categorical assignments (Heston, Thomas F and Gillette, Justin 2008; Hüllermeier et al. 2025). This ranking-centric approach directly addresses the core challenge of modeling dimensional interdependencies by conceptualizing personality assessment as a continuous spectrum rather than discrete categories. Implementing this paradigm shift, we introduce PerDet-R1, a two-stage framework that eliminates categorical fallacies through ranking-based modeling. In the first stage, we perform supervised fine-tuning (SFT) by distilling comparative reasoning traces from advanced LLMs Qwen-plus to generate ground-truth-aligned MBTI preference rankings from social media data, training a 7B-parameter LLM to output ordered type lists with explicit awareness of dimensional combinations and interactions. The second stage implements Group Relative Policy Optimization (GRPO), initializing the policy with our stage-one model and optimizing ranking positions via a Normalized Discounted Cumulative Gain (NDCG) reward function, thereby amplifying trait-cohort interactions through position-sensitive reinforcement while eliminating expert prompt dependency. In summary, our core contributions are as follows: • We propose the first list-wise ranking paradigm for personality detection, which eliminates the categorical fallacy inherent in existing methods. • We introduce a trait-cohort advantage estimation for the reward model, which calculates the reward score by dimension similarity and NDCG to amplify subtle interactions, generating dense reward signals to effectively avoid the problem of reward hacking. • Our method achieves state-of-the-art results on mainstream personality detection datasets, establishing a new standard for data-driven personality assessment. Related Work Personality Detection Pre-trained language models (e.g., BERT, RoBERTa) dominated personality detection before the emergence of large language models (LLMs). While demonstrating performance gains through encoding concatenated user utterances, these approaches neglected temporal dynamics and inter-post relationships (Tandera et al. 2017; Xue et al. 2018; Lynn, Balasubramanian, and Schwartz 2020). To address the limitations, Yang et al. (2023a) address the unstructured post-integration challenge in personality detection through Deep Dynamic Graph Convolutional Network (D-DGCN). Departing from prior graph-based methods that require predefined adjacency structures. Concurrently with architectural innovations, LLM-based reasoning techniques emerged, and Chain-of-Thought (CoT) prompting has been introduced to steer LLMs in generating explicit reasoning pathways. Building upon standard CoT, PsyCoT (Yang et al. 2023b) innovatively adapts psychological questionnaires into a structured CoT framework. This approach simulates human self-assessment procedures via multi-turn dialogues, directing LLMs to focus on specific test items while leveraging historical ratings for more consistent and accurate reasoning. Hu et al. (2024) address the data scarcity challenge in personality detection through LLM-based text augmentation. Their method distills LLM capabilities (comprehension, summarization, sentiment analysis) to generate multi-perspective text analyses, semantic, affective, and linguistic. Bi et al. (2025) propose the LLM-Enhanced Text Mapping Model (ETM), a hybrid architecture addressing core limitations in traditional personality detection systems. ETM demonstrates how LLMs can dynamically construct psychologically-grounded representations. Although existing methods achieve notable success, they universally adopt classification-centric frameworks, either selecting among 16 discrete MBTI types or aggregating binary predictions across four dimensions. These approaches fundamentally diverge from our work, which harnesses LLMs’ reasoning capabilities to reframe personality detection as comparative ranking. By amplifying inter-type distinctions through relative intensity assessments, our paradigm captures nuanced trait interactions obscured by classification methods, thereby advancing detection performance. Fine-Tuning Language Models Beyond architectural innovations, training strategies also play a crucial role in enhancing personality detection capabilities (Wang and Sun 2024; Serapio-García et al. 2025). Fine-tuning has emerged as a pivotal strategy for adapting language models to specialized tasks (Shen et al. 2025). Recent innovations integrate explicit reasoning traces during fine-tuning, where step-by-step rationales guide models through complex problem-solving (e.g., mathematics or coding) (Wei et al. 2022). These traces mitigate hallucination risks and enhance logical coherence, which proves essential for reasoning scenarios. Reinforcement Learning (RL) further augments reasoning capabilities. Algorithms like Group Relative Policy Optimization (GRPO) (Guo et

Read the complete text