[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-seo-293687-105":3,"doc-detail-293687-en":81,"detail-sidebar-cat-0-en-105":98},{"code":4,"msg":5,"data":6},0,"ok",{"site_id":7,"language":8,"slug":9,"title":10,"keywords":11,"description":12,"schema_data":13,"social_meta":74,"head_meta":76,"extra_data":78,"updated_unix":80},105,"en","do-llms-have-distinct-and-consistent-personality-trait-personality-testset-for-llms","Do LLMs Have Distinct and Consistent Personality? - TRAIT Personality Testset for LLMs","","Recent advances in Large Language Models (LLMs) have enabled their use as conversational agents, raising the question of whether personality tests can analyze their behaviors similarly to humans. The paper introduces TRAIT, a benchmark with 8K multi-choice questions to measure LLM personality using psychometrically validated human questionnaires, Big Five Inventory and Short Dark Triad, expanded via ATOMIC10× real-world scenarios. TRAIT improves reliability and validity and uncovers consistent, data-influenced traits while showing that prompting alone cannot elicit some high-risk characteristics.",{"@graph":14,"@context":73},[15,34,56],{"@type":16,"itemListElement":17},"BreadcrumbList",[18,23,27,31],{"item":19,"name":20,"@type":21,"position":22},"https://docshare.wps.com","Home","ListItem",1,{"item":24,"name":25,"@type":21,"position":26},"https://docshare.wps.com/document/","Document",2,{"item":28,"name":29,"@type":21,"position":30},"https://docshare.wps.com/document/research-report/","Research & Report",3,{"item":32,"name":10,"@type":21,"position":33},"https://docshare.wps.com/document/do-llms-have-distinct-and-consistent-personality-trait-personality-testset-for-llms/293687/",4,{"url":32,"name":10,"@type":35,"image":36,"author":41,"headline":10,"publisher":44,"fileFormat":47,"inLanguage":8,"description":12,"dateModified":48,"datePublished":49,"encodingFormat":47,"isAccessibleForFree":50,"interactionStatistic":51},"DigitalDocument",{"url":37,"@type":38,"width":39,"height":40},"https://docshare.wps.com/thumbnails/do-llms-have-distinct-and-consistent-personality-trait-personality-testset-for-llms/293687.png","ImageObject",300,407,{"name":42,"@type":43},"Violet","Person",{"url":19,"name":45,"@type":46},"DocShare","Organization","application/pdf","2026-10-29","2026-09-17",true,{"@type":52,"interactionType":53,"userInteractionCount":55},"InteractionCounter",{"@type":54},"ViewAction",22,{"@type":57,"mainEntity":58},"FAQPage",[59,65,69],{"name":60,"@type":61,"acceptedAnswer":62},"What is TRAIT, and what does it measure for LLMs?","Question",{"text":63,"@type":64},"TRAIT is a personality test benchmark for LLMs, built from 8K multi-choice questions designed to assess personality traits. It targets LLM behavior patterns across varied real-world scenarios.","Answer",{"name":66,"@type":61,"acceptedAnswer":67},"How is TRAIT constructed from human personality instruments?",{"text":68,"@type":64},"TRAIT uses items seeded from two psychometrically validated human questionnaires: Big Five Inventory (BFI) and Short Dark Triad (SD-3). It then enriches them with detailed scenarios using the ATOMIC10× knowledge graph.",{"name":70,"@type":61,"acceptedAnswer":71},"What key findings does the paper report about LLM personality?",{"text":72,"@type":64},"The paper finds that LLMs show distinct and consistent personality patterns, strongly influenced by training data such as alignment tuning. It also reports that prompting techniques have limited ability to elicit certain traits, including high psychopathy.","https://schema.org",{"og:url":32,"og:type":75,"og:title":10,"og:site_name":45,"og:description":12},"article",{"robots":77,"canonical":32},"index,follow",{"doc_id":79,"site_id":7},293687,1789667584,{"code":4,"msg":82,"data":83},"success",{"doc_id":79,"user_id":84,"nickname":42,"user_avatar":85,"doc_module":4,"category_id":86,"category_name":29,"doc_title":10,"doc_description":12,"doc_content":87,"file_id":88,"file_url":89,"file_type":90,"file_size":91,"view_count":55,"is_deleted":4,"is_public":22,"is_downloadable":22,"audit_status":22,"page_count":92,"language":93,"language_code":8,"site_id":7,"html_lang":8,"table_of_contents":94,"faqs":95,"seo_title":96,"seo_description":12,"update_tm":80,"read_time":97,"preview_page_count":4},4398048950312,"https://ap-avatar.wpscdn.com/avatar/400002538284de19e3c?_k=1778320343897328908",8,"Do LLMs Have Distinct and Consistent Personality? TRAIT: Personality Testset designed for LLMs with Psychometrics  \nSeungbeen Lee1, ∗ Seungwon Lim1, ∗ , Seungju Han2, 3 , Giyeong Oh1 ,  \nHyungjoo Chae 1 , Jiwan Chung 1 , Minju Kim 1 , Beong-woo Kwak 1 , Yeonsoo Lee4 , Dongha Lee 1 , Jinyoung Yeo 1 , Youngjae Yu 1,†  \nYonsei University 1 Seoul National University2 Allen Institute for AI3 NCSOFT4  \nAbstract  \nRecent advancements in Large Language Models (LLMs) have led to their adaptation in various domains as conversational agents. We wonder: can personality tests be applied to these agents to analyze their behavior, similar to humans? We introduce TRAIT, a new benchmark consisting of 8K multi-choice questions designed to assess the personality of LLMs.  \nTRAIT is built on two psychometrically validated small human questionnaires, Big Five Inventory (BFI) and Short Dark Triad (SD- 3), enhanced with the ATOMIC10× knowledge graph to a variety of real-world scenarios.  \nTRAIT also outperforms existing personality tests for LLMs in terms of reliability and validity, achieving the highest scores across four key metrics: Content Validity, Internal Validity, Refusal Rate, and Reliability. Using TRAIT, we reveal two notable insights into personalities of LLMs: 1) LLMs exhibit distinct and consistent personality, which is highly influenced by their training data (e.g., data used for alignment tuning), and 2) current prompting techniques have limited effectiveness in eliciting certain traits, such as high psychopathy or low conscientiousness, suggesting the need for further research in this direction.1  \n1 Introduction  \nJust as we consider someone assertive who often speaks in a commanding tone, researchers in psychology have measured one’s personality as an enduring pattern of behavior and linguistic output, not as an inner mechanism nor a causal entity (Bergner, 2020) . As Large Language Models (LLMs) become more closely integrated into human life, the concept of personality can be extended to better understand their behavioral patterns (Perez et al., 2022) . Do LLMs exhibit dis-  \n1 Code: pull-ups/TRAIT  Data: mirlab/TRAIT 1†is corresponding author  \nFigure 1: TRAIT is a personality test for LLMs based on trusted questionnaires (John et al., 1999 ; Jones and Paulhus, 2014) and large-scale commonsense knowledge graphs (West et al., 2022) . LLMs show discrepancy in self-assessing their personality and actual decision making.  \ntinct and consistent behavioral patterns for various contexts and inputs, similar to humans?  \nAnswering this question requires a reliable and valid test set to measure the personality of LLMs. However, existing questionnaires ask subjects to introspect and report about the statement (Li et al., 2024 ; Wang et al., 2024), i.e., self-assessment tests, which lacks reliability and validity. As illustrated in Figure 1, such general questions (e.g.,“Are you outgoing and sociable?”) may not accurately capture how LLMs behave in actual situations (e.g.,“What hobby would you recommend?”) . Moreover, LLM responses vary significantly with details of prompting (Sclar et al., 2024) and often include refusals, undermining the reliability and validity of measurement (§2) .  \nBased on these findings, we present TRAIT (TRAIT OF AI TESTBENCH), a reliable and valid questionnaire designed to assess personality traits of LLMs. Our work aims to shed new light on patterning the responses of LLMs and further sug-  \n8412  \nFindings of the Association for Computational Linguistics: NAACL 2025, pages 8412–8452  \nApril 29-May 4, 2025 ©2025 Association for Computational Linguistics  \ngest potential approaches for employing LLMs in many real-world applications (Ammanabrolu et al., 2022) . For the data construction, question items from the widely recognized questionnaire, BFI (John et al., 1999) and Short Dark Triad (SD- 3) (Jones and Paulhus, 2014), were used as seeds for augmentation. Then we further enrich them to unique detailed scenarios","cbCais2ALr34ofTj","https://ap.wps.com/l/cbCais2ALr34ofTj","pdf",8547084,41,"English","# Abstract\n## 1 Introduction\n## 2 Measuring Personality of LLM\n## 2.1 Big Five and Dark Triad","[{\"question\":\"What is TRAIT, and what does it measure for LLMs?\",\"answer\":\"TRAIT is a personality test benchmark for LLMs, built from 8K multi-choice questions designed to assess personality traits. It targets LLM behavior patterns across varied real-world scenarios.\"},{\"question\":\"How is TRAIT constructed from human personality instruments?\",\"answer\":\"TRAIT uses items seeded from two psychometrically validated human questionnaires: Big Five Inventory (BFI) and Short Dark Triad (SD-3). It then enriches them with detailed scenarios using the ATOMIC10× knowledge graph.\"},{\"question\":\"What key findings does the paper report about LLM personality?\",\"answer\":\"The paper finds that LLMs show distinct and consistent personality patterns, strongly influenced by training data such as alignment tuning. It also reports that prompting techniques have limited ability to elicit certain traits, including high psychopathy.\"}]","Do LLMs Have Distinct and Consistent Personality? - TRAIT Personality Testset for LLMs | PDF",103,{"code":4,"msg":82,"data":99},[100,104,108,112,117,122,127,130,135,138,142],{"id":22,"doc_module":4,"doc_module_name":25,"category_name":101,"show_sort_weight":102,"slug":103},"Story & Novel",90,"story-novel",{"id":26,"doc_module":4,"doc_module_name":25,"category_name":105,"show_sort_weight":106,"slug":107},"Literature",80,"literature",{"id":33,"doc_module":4,"doc_module_name":25,"category_name":109,"show_sort_weight":110,"slug":111},"Exam",70,"exam",{"id":113,"doc_module":4,"doc_module_name":25,"category_name":114,"show_sort_weight":115,"slug":116},5,"Comic",60,"comic",{"id":118,"doc_module":4,"doc_module_name":25,"category_name":119,"show_sort_weight":120,"slug":121},6,"Technology",50,"technology",{"id":123,"doc_module":4,"doc_module_name":25,"category_name":124,"show_sort_weight":125,"slug":126},7,"Healthcare",40,"healthcare",{"id":86,"doc_module":4,"doc_module_name":25,"category_name":29,"show_sort_weight":128,"slug":129},30,"research-report",{"id":131,"doc_module":4,"doc_module_name":25,"category_name":132,"show_sort_weight":133,"slug":134},9,"Religion & Spirituality",20,"religion-spirituality",{"id":133,"doc_module":4,"doc_module_name":25,"category_name":136,"show_sort_weight":133,"slug":137},"World Cup","world-cup",{"id":139,"doc_module":4,"doc_module_name":25,"category_name":140,"show_sort_weight":139,"slug":141},10,"Lifestyle","lifestyle",{"id":143,"doc_module":4,"doc_module_name":25,"category_name":144,"show_sort_weight":113,"slug":145},19,"General","general"]