Cognitive and Behavioral Human-Machine Alignment - From Individuals to Collectives - Thesis

Sep 17, 2026

Uploaded by Theodore

Large Language Models (LLMs) demonstrate strong human-like capabilities, prompting questions about whether they can show personality, temperament, and emotion-like behavior. This thesis develops a comprehensive evaluation framework using two perspectives: individual psychological profiling and collective social-context evaluation. It validates reliable personality assessment proxies, introduces PsychoBench and EmotionBench for psychological and emotional alignment testing, and proposes GAMA(γ)-Bench for robust game-theoretic multi-agent strategy evaluation. Finally, it studies resilience against malicious agents and presents defensive mechanisms to mitigate harmful influence.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages296

Reading Time12 h 26 min

AccessFree to download

Abstract

Individual Perspective: Personality and Emotional Responses

PsychoBench

EmotionBench

Collective Perspective: Social, Competitive, and Collaborative Behavior

GAMA(γ)-Bench

Multi-agent Resilience and Defense Mechanisms

Cognitive and Behavioral Human-Machine Alignment: From Individuals to Collectives HUANG, Jen Tse A Thesis Submitted in Partial Fulfilment of the Requirements for the Degree of Doctor of Philosophy in Computer Science and Engineering The Chinese University of Hong Kong February 2025 Thesis Assessment Committee Professor Farzan FARNIA (Chair) Professor LYU Rung Tsong Michael (Thesis Supervisor) Professor WANG Liwei (Committee Member) Professor MA Lei (External Examiner) Abstract Large Language Models (LLMs) have recently showcased their remarkable capacities, not only in natural language processing tasks but also across diverse domains such as clinical medicine, legal consultation, and education. They have become more than mere applications, evolving into assistants capable of addressing diverse user requests. This narrows the distinction between human beings and artificial intelligence agents, raising intriguing questions regarding the potential manifestation of personalities, temperaments, and emotions within LLMs. To advance our understanding of the human-like capabilities of LLMs, this thesis introduces a comprehensive evaluation framework encompassing two core perspectives. The Individual perspective examines independent LLM entities to assess their psychological profiles and emotional responses to environmental stimuli. Conversely, the Collective perspective evaluates LLM behavior in social contexts, such as competition in game-theoretic scenarios and collaboration in shared tasks. We first assess the reliability of personality assessments on LLMs, demonstrating that models like GPT-3.5, GPT-4, and LLaMA-3.1 exhibit consistent personality traits. These findings lay the groundwork for employing LLMs as proxies for human participants, offering a cost-effective alternative for behavioral research. Expanding on this, we introduce PsychoBench, a comprehensive framework utilizing thirteen psychological scales, to profile LLMs across personality, interpersonal, motivational, and emotional domains, while examining model behaviors via jailbreak techniques to better understand intrinsic responses outside safety protocols. We further examine LLMs’ anthropomorphic potential, specifically in emotional alignment with human responses. Using a novel framework, EmotionBench, we assess how LLMs respond to a dataset of emotionally evocative scenarios, uncovering limitations in LLMs’ empathetic alignment, particularly in associating similar situations with consistent emotional responses. Next, in the context of decision-making, we introduce GAMA(γ)-Bench, a multi-agent framework based on game theory that enables robust evaluation of LLMs’ competitive strategies across dynamic scenarios. Findings show that while GPT-3.5 displays strong robustness, Gemini-1.5-Pro leads in overall performance and adaptability in strategic environments. Finally, we analyze the resilience of multi-agent systems against malicious agents, demonstrating that hierarchical structures provide superior robustness with lower performance degradation under adversarial influence. By proposing two defensive mechanisms—cross-agent challenge and independent review agents—we enhance the system’s ability to mitigate harmful influences. Collectively, these contributions advance our understanding of LLMs’ psychological, emotional, and collaborative dynamics, with implications for their deployment in socially aligned and resilient multi-agent systems. 摘要大型語言模型（LLMs）近期展現出其非凡的能力，不僅在自然語言處理任務中表現優異，還在臨床醫學、法律諮詢及教育等多個領域取得了顯著成效。它們不再僅僅是應用程序，正逐步演變為能夠應對各種用戶需求的助手。這縮小了人類與人工智能代理之間的差距，並引發了關於 LLMs 是否可能表現出人格、氣質與情感的有趣問題。為進一步理解 LLMs 的擬人化能力，本論文提出了一套全面的評估框架，涵蓋兩個核心視角。個體視角針對單獨的 LLM 實體進行評估，以分析其心理特徵及其對環境刺激的情緒反應。相比之下，集體視角則聚焦於 LLMs 在社會情境中的行為表現，例如在博弈論場景中的競爭和在共同任務中的協作。首先，我們評估了人格評估在 LLMs 上的可靠性，結果顯示 GPT-3 .5、 GPT-4 和 LLaMA-3 .1 等模型展現出穩定的人格特徵，為使用 LLMs 替代人類參與者進行行為研究奠定了基礎，並提供了一種具成本效益的替代方案。在此基礎上，我們引入了 PsychoBench，一個涵蓋十三種心理量表的綜合框架，用於分析 LLMs 在性格、社交、動機和情感領域的表現，同時通過“越獄”技術探索模型在安全協

FAQs

What two perspectives does the thesis use to evaluate LLM behavior?

It uses an individual perspective to assess independent LLM entities’ psychological profiles and emotional reactions, and a collective perspective to evaluate LLM behavior in social contexts such as competition and collaboration.

How does the thesis assess personality and emotional alignment in LLMs?

It first checks reliability of personality assessments and then introduces PsychoBench for multidomain psychological profiling and EmotionBench to evaluate emotional responses to emotionally evocative scenarios.

What is GAMA(γ)-Bench used for, and what key findings are reported?

GAMA(γ)-Bench is a multi-agent game-theoretic framework for evaluating LLM competitive strategies in dynamic scenarios; results indicate strong robustness for GPT-3.5 and overall lead for Gemini-1.5-Pro in performance and adaptability.

How does the thesis improve multi-agent system resilience against malicious agents?

It analyzes robustness of hierarchical multi-agent structures and proposes two defensive mechanisms—cross-agent challenge and independent review agents—to mitigate harmful influences under adversarial conditions.

Sep 17, 2026

Uploaded by Theodore