Large Language Models as a Judge in Theory-Agnostic Adaptive Metric-Alignment for Prototypical Networks in Personality Recognition

Jul 16, 2026

Personality recognition often relies on theory-dependent formulations that fit predefined psychological taxonomies rather than revealing shared underlying structure, which limits cross-framework generalization. The JAM framework (Judge for Adaptive Metric-Alignment) shifts training from adapting to fixed personality theories to discovering unified latent “pseudo-facets” from text samples without theory-specific labels. JAM builds structured representations using an attention-pooled graph prototypical network and cross-theory harmonization, and improves robustness with LLM-as-a-judge in two operating modes that detect ambiguous or mislabeled boundary cases.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages16

Reading Time40 min

Introduction

Contributions

Large-Language-Models-as-a-Judge in Theory-Agnostic Adaptive Metric-Alignment for Prototypical Networks in Personality Recognition Jing Jie Tan∗ , Ban-Hoe Kwan∗ , Danny Wee-Kiat Ng∗ , Yan-Chai Hum∗ , Shih-Yu Lo †, Po-An Chen ‡, Noriyuki Kawarazaki§ , Kosuke Takano§ , Anissa Mokraoui¶ ∗Department of Mechatronics and Biomedical Engineering, Lee Kong Chian Faculty of Engineering and Science, Universiti Tunku Abdul Rahman, Malaysia †Institute of Communication Studies, National Yang Ming Chiao Tung University, Taiwan ‡Institute of Information Management, National Yang Ming Chiao Tung University, Taiwan §Faculty of Information Technology, Kanagawa Institute of Technology, Japan ¶Laboratoire de Traitement et Transport de l’Information, Université Sorbonne Paris Nord, France Email: [tanjingjie@1utar.my](tanjingjie@1utar.my), {kwanbh, ngwk, [humyc}@utar.edu.my](humyc}@utar.edu.my),{shihyulo, [poanchen}@nycu.edu.tw](poanchen}@nycu.edu.tw),{kawara@rm, [takano@ic}.kanagawa-it.ac.jp](takano@ic}.kanagawa-it.ac.jp), [anissa.mokraoui@univ-paris13.fr](anissa.mokraoui@univ-paris13.fr) arXiv :2607 .08374v 1 [ cs .CL] 9 Jul 2026 Abstract—Personality recognition has traditionally been constrained by theory-dependent formulations, where models are trained to fit predefined psychological taxonomies rather than uncovering shared underlying behavioral structure. This limits generalization, as personality itself is better understood as theory-invariant, emerging from stable psychological patterns that should manifest consistently across different frameworks, while existing annotations reflect only partial and sometimes inconsistent views of the same latent traits. In this work, we introduce JAM ((J)udge for (A)daptive (M)etric-Alignment), a theory-agnostic framework that shifts learning from adapting to predefined personality theories toward discovering unified latent“pseudo-facets” that capture shared psychological structure. Rather than constraining the model to any personality taxonomy during training or inference, the framework learns generalizable psychological representations and can infer an individual’s latent psychological profile directly from the textual samples, without requiring theory-specific labels. JAM achieves this through an Attention-Pooled Graph Prototypical Network that learns structured representations via clustering in embedding space, together with a Cross-Theory Harmonization (CTH) approach that integrates (i) Human-Guided Linkage and (ii) MachineInduced Consensus to unify heterogeneous datasets without relying on predefined labels. To further improve robustness and data quality, we incorporate an LLM-as-a-Judge mechanism operating in two configurations, (i) LLM-before-the-loop and (ii) LLM-in-the-loop which identifies ambiguous, mislabeled, and boundary samples to guide adaptive metric learning. Experiments on Essays and Kaggle personality datasets show that JAM improves cross-framework generalization and performance, establishing a strong step toward theory-agnostic personality inference and supporting low-resource personality theories. The related code repository, model weights, and artifacts are available at [https://research.jingjietan.com/JAM](https://research.jingjietan.com/JAM). Index Terms—Personality Classification, Large Language Models (LLMs), N-Shot Prompting, Prototypical Networks, FineTuning, Natural Language Understanding I. INTRODUCTION Personality recognition has become increasingly important, especially in recommendation systems [1] . By understanding user personalities, these systems can provide personalized suggestions, enhancing user satisfaction and trust. Understanding user personality is crucial for delivering a superior user experience, making this an important area of study [2] . By tailoring interactions and recommendations to individual personality traits, AI systems and robots can achieve higher levels of personalization, leading to increased user satisfaction and trust [3]

FAQs

Why do traditional personality recognition models struggle to generalize across datasets and cultures?

They are typically constrained by theory-dependent formulations (e.g., Big-5 or MBTI), and rely on scarce annotated data, which reduces generalization across frameworks and cultural contexts.

What is JAM and what does it change about the learning objective?

JAM (Judge for Adaptive Metric-Alignment) is a theory-agnostic framework that replaces reliance on predefined personality theories with learning generalizable latent “pseudo-facets” from text samples without requiring theory-specific labels.

How does JAM integrate heterogeneous datasets across different personality theories?

It uses an Attention-Pooled Graph Prototypical Network for structured embedding representations and a Cross-Theory Harmonization (CTH) approach combining human-guided linkage and machine-induced consensus to unify heterogeneous data.

Jul 16, 2026

Upload by Valentina