Identifying and Mitigating Bottlenecks in Role-Playing Agents - A Systematic Study of Disentangling Character Profile Axes

Sep 17, 2026

Uploaded by Đào

Large Language Model (LLM) roleplaying agents advance quickly, yet the character profile elements that truly govern role-playing quality remain unclear. This study introduces a systematic diagnostic framework that separates profile effects across Familiarity (Known vs. Unknown), Structure (Structured vs. Unstructured), and Disposition (Moral vs. Immoral). Using a unified hierarchical schema with 5 dimensions and 28 fields, the work builds 211 personas and evaluates five LLMs on single- and multi-turn interactions. Results show Disposition drives consistent performance degradation, amplified by post-SFT alignment, while Familiarity and Structure are negligible. The proposed Field-Aware Contrastive Decoding (FACD) mitigates the bottleneck in a training-free way, closing the gap without harming moral-character performance.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages31

Reading Time1 h 18 min

AccessFree to download

Introduction

Role of character profiles in RPAs

Three-axis diagnostic framework (Familiarity, Structure, Disposition)

Unified hierarchical profile schema and controlled dataset

Research questions (RQ1, RQ2)

Identifying and Mitigating Bottlenecks in Role-Playing Agents: A Systematic Study of Disentangling Character Profile Axes Yonghyun Jun* Junhyuk Choi* Jeonghyun Park Jihyeong Park Liu Nicole Geumheon Hwanhee Lee† Chung-Ang University, Seoul, Korea {zgold5670,chlwnsgur129,tom0365,g2hyeong,febygh, [hwanheelee}@cau.ac.kr](hwanheelee}@cau.ac.kr)[ ](hwanheelee}@cau.ac.kr)[https://github.com/yonghyun010102/RPA_Bottleneck.git](https://github.com/yonghyun010102/RPA_Bottleneck.git) arXiv : 260 1 .047 16v4 [cs .CL] 3 1 Aug 2026 Abstract While Large Language Model (LLM) roleplaying agents have advanced rapidly, it remains unclear which profile elements genuinely drive role-playing quality. To bridge this gap, we introduce a systematic diagnostic framework that disentangles the impact of character profiles along three axes: Familiarity (Known vs. Unknown), Structure (Structured vs. Unstructured), and Disposition (Moral vs. Immoral) . Utilizing a unified hierarchical schema (5 dimensions, 28 fields), we construct a controlled dataset of 211 personas and evaluate five LLMs on both single-and multi-turn interactions. Our results reveal a striking asymmetry: Familiarity and Structure show negligible impact, while Disposition produces large, consistent performance degradation for immoral characters across all conditions. Further analyses suggest that the Moral–Immoral gap is amplified by post-SFT alignment, and that this degradation varies substantially across profile attributes. To mitigate this bottleneck, we propose Field-Aware Contrastive Decoding (FACD), a training-free strategy that amplifies suppressed disposition-sensitive signals, significantly closing the performance gap without sacrificing moral-character performance. 1 Introduction Advancements in Large Language Models (LLMs) have catalyzed a surge of interest in Role-Playing Agents (RPAs) (Park et al., 2023 ; Chuang et al., 2024 ; Wang et al., 2024c ; Li et al., 2023a ; Wanget al., 2024a) . Alongside rapid industrial adoption by character-based chatting services such as Character.AI* , extensive research efforts have focused on improving RPA performance through various approaches, including reasoning (Chen et al., 2024), retrieval-augmented generation (Huang et al., 2024), and fine-tuning (Sun et al., 2025) . * Equal contribution.† Corresponding author. *[https://character.ai/](https://character.ai/) | Pre-training
Source

RPA | (A) Familiarity

Pre-training
Source

RPA | | --- | --- | | | | | Tuning Template
{ … …
… }

RPA
(B) Structure
Structured Character Profile Unstructured Character Profile

Tuning Template
{ … …
… }

RPA | | | | | | Model
Alignment

RPA | (C) Disposition Model Alignment
| Figure 1: The three axes of character profile: (A) Familiarity: Known vs. Unknown,(B) Structure: Structured vs. Unstructured, and (C) Disposition: Moral vs. Immoral characters. Central to these systems is the character profile, the foundational specification dictating the portrayed persona. RPA performance can fluctuate significantly based on what attributes are selected and how they are presented as a prompt. However, existing works fail to systematically disentangle these factors under controlled conditions; they exclusively target either Known (Wang et al., 2025a ; Liu et al., 2024) or Unknown characters (Wanget al., 2025b ; Zhou et al., 2025), employ ad-hoc templates, and rarely isolate individual traits. Consequently, it remains unclear whether an agent’s fidelity is driven by a character’s intrinsic nature or its representational format. In this paper, we conduct a systematic diagnostic study to determine how specific attribute configurations and representational methods drive roleplaying performance. To ground our analysis in LLM development mechanisms, we hypothesize that profile impacts can be categorized into thre

FAQs

Which character profile axes most strongly affect role-playing agent quality?

The study finds a clear asymmetry: Familiarity and Structure show negligible impact, while Disposition causes large, consistent performance degradation for immoral characters across conditions.

How does the paper evaluate the effect of character profiles?

It builds a controlled dataset of 211 personas using a unified hierarchical schema (5 dimensions, 28 fields) and evaluates five LLMs on single- and multi-turn interactions.

What causes the moral–immoral performance gap to become larger?

Further analyses suggest the Moral–Immoral gap is amplified by post-SFT alignment, with degradation varying substantially across profile attributes.

Sep 17, 2026

Uploaded by Đào