Can Large Language Models Understand You Better? An MBTI Personality Detection Dataset Aligned with Population Traits

Sep 17, 2026

Uploaded by Ezra

The Myers-Briggs Type Indicator (MBTI) models individual differences in thinking, feeling, and behavior, and MBTI personality detection has drawn extensive research attention. However, existing datasets often over-assume ideal alignment with real population trait distributions: self-reported labels introduce incorrect labeling, and hard labels cannot represent the continuum of traits. MBTIBENCH is presented as a manually annotated, psychologist-guided dataset with soft labels and improved quality, addressing both challenges and revealing polarized LLM behavior plus advantages of soft labels.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages28

Reading Time1 h 11 min

AccessFree to download

Abstract

Introduction

MBTI Personality Detection Dataset

arXiv :2412 . 12510v1 [ cs .CL] 17 Dec 2024 Can Large Language Models Understand You Better? An MBTI Personality Detection Dataset Aligned with Population Traits Bohan Li1, Jiannan Guan1, Longxu Dou2, Yunlong Feng1, Dingzirui Wang1, Yang Xu1, Enbo Wang1, Qiguang Chen1, Bichen Wang1, Xiao Xu1, Yimeng Zhang2, Libo Qin3, Yanyan Zhao1, Qingfu Zhu1, Wanxiang Che1 ∗ 1 Harbin Institute of Technology 2 Individual Researcher 3 Central South University {bhli, [jnguan](jnguan}@ir.hit.edu.cn)[}](jnguan}@ir.hit.edu.cn)[@ir.hit.edu.cn](jnguan}@ir.hit.edu.cn), [car@ir.hit.edu.cn](car@ir.hit.edu.cn)* Abstract The Myers-Briggs Type Indicator (MBTI) is one of the most influential personality theories reflecting individual differences in thinking, feeling, and behaving. MBTI personality detection has garnered considerable research interest and has evolved significantly over the years. However, this task tends to be overly optimistic, as it currently does not align well with the natural distribution of population personality traits. Specifically,(1) the self-reported labels in existing datasets result in incorrect labeling issues, and (2) the hard labels fail to capture the full range of population personality distributions. In this paper, we optimize the task by constructing MBTIBENCH, the first manually annotated high-quality MBTI personality detection dataset with soft labels, under the guidance of psychologists. As for the first challenge, MBTIBENCH effectively solves the incorrect labeling issues, which account for 29.58% of the data. As for the second challenge, we estimate soft labels by deriving the polarity tendency of samples. The obtained soft labels confirm that there are more people with non-extreme personality traits. Experimental results not only highlight the polarized predictions and biases in LLMs as key directions for future research, but also confirm that soft labels can provide more benefits to other psychological tasks than hard labels.1 1 Introduction Personality, a key psychological concept, refers to individual differences in thinking, feeling, and behavior (Corr & Matthews, 2009) . Among personality models, the Myers-Briggs Type Indicator (MBTI) is one of the most recognized non-clinical frameworks, with broad applications in areas like stance detection (Hosseinia et al., 2021) . Recently, automatically detecting a person’s MBTI type from their written content (e.g., tweets and blogs) (Plank & Hovy, 2015; Gjurkovi et al., 2021) has become a growing area of research with both academic significance (Stajner & Yenikent, 2020; Khan et al., 2005) and practical applications (Bagbyet al., 2016) . However, MBTI personality detection tends to be overly optimistic, as it currently does not align well with the natural distribution of personality traits within the population. (1) Given the widespread use of MBTI, existing datasets are often derived from social media posts, with labels typically sourced from users’ self-reported MBTI types (Gjurkovi et al., 2021) . Assessments conducted by non-professional psychological institutions, along with ∗ The corresponding author. 1 [https://github.com/Personality-NLP/MbtiBench](https://github.com/Personality-NLP/MbtiBench) inaccurate self-perception, can lead to a mismatch between the self-reported personality traits and the actual linguistic patterns in the text (McDonald, 2008; Mller & Moshagen, 2019; Paulhus & Vazire, 2007) . For example, a user who self-identifies as an Extraversion type might exhibit more Introversion traits in their posts (Figure 9 (a)) . (2) From a psychological standpoint, personality is not binary but rather complex, nuanced, and highly individualized (Wu et al., 2022) . Most people don’t display extreme personality traits; instead, they tend to fall somewhere in the middle (Tzeng et al., 1989) . However, existing datasets use only binary MBTI labels, missing the full spectrum of personality traits in most people (Harvey & Murry, 1994; Wu

FAQs

What main problem do existing MBTI personality detection datasets have?

They often fail to align with real population trait distributions because self-reported labels can be inaccurate and hard labels do not capture the full spectrum of traits.

How does MBTIBENCH address incorrect labeling?

It introduces data filtering guidelines and re-annotates the cleaned datasets under psychological expert guidance to better match personality polarity.

Why are soft labels important in MBTIBENCH?

Soft labels are estimated from polarity tendencies, reflecting that more people have non-extreme traits; results also show soft labels benefit related psychological tasks more than hard labels.

Sep 17, 2026

Uploaded by Ezra