Prompt Optimization for User Simulation in Conversational Recommender Systems: A Multi-Objective Framework

Jul 16, 2026

Conversational recommender systems enable users to elicit preferences, clarify intentions, and receive adaptive recommendations in real time, but evaluation and training face two major barriers: real-user studies are costly and slow, and conversational training data is hard to obtain because of privacy constraints. LLM-based user simulators mitigate these issues by generating synthetic interactions for evaluation and training. Existing simulators, however, suffer from positive bias, data leakage, and limited behavioral diversity caused by brittle manual prompt engineering. A framework is proposed to automatically optimize prompts while addressing these problems and improving behavioral alignment with human interaction patterns across diverse settings.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages10

Reading Time25 min

Introduction

Challenges in Current LLM-Based User Simulation

Types of Prompt-Based vs Fine-Tuned Simulators

Prompt Optimization for User Simulation in Conversational Recommender Systems: A Multi-Objective Framework Nipun B Nair Monash University, Australia[nipun.nair@monash.edu](nipun.nair@monash.edu) Tongtong Wu Monash University, Australia [tongtong.wu@monash.edu](tongtong.wu@monash.edu) Weiqing Wang Monash University, Australia [teresa.wang@monash.edu](teresa.wang@monash.edu) arXiv :2607 .000 10v 1 [ cs .IR] 8 May 2026 Abstract—Conversational recommender systems (CRSs) area core component of next-generation intelligent recommender systems because they enable users to actively elicit preferences, clarify intentions, and adapt recommendations in real time. However, there are two key obstacles in the CRS domain: evaluation and access to training data. Evaluating CRSs through real human studies is more critical than for traditional recommender systems, yet such studies are both costly and time-consuming. Moreover, CRS interaction data are often difficult to obtain for model training due to privacy concerns. Large language model (LLM) -based user simulators have shown promise in addressing both challenges by generating synthetic user interactions for evaluation and training. However, existing approaches suffer from systematic positive bias, data leakage, and limited behavioral diversity, and they rely on brittle manual prompt engineering that requires extensive domain expertise. In this paper, we propose a framework to automatically optimize prompts for LLM-based user simulators in CRSs, simultaneously mitigating these issues. Experimental results demonstrate that the proposed framework achieves improved behavioral alignment with human interaction patterns compared to baseline methods across diverse prompt settings. Index Terms—User simulation, Prompt tuning, Conversational Recommendation Systems, LLMs I. INTRODUCTION Recommender systems [1], [2] play a critical role in information seeking by enabling users to efficiently discover relevant items, content, and information. These systems deliver both user value and business impact and continue to drive strong academic and industrial interest [1], [3]–[5] . Conversational recommender systems (CRSs) enhance this capability by engaging users in natural language to infer user preferences and reasons behind those preferences. CRSs allows users to articulate preferences, explore options interactively, and provide fine-grained feedback [6], [7] which is fundamentally different from traditional recommender systems and have been recognized as the core component of next-generation intelligent recommender systems in this Large Language Model (LLM) era [6], [8], [9] . Conversational recommender systems typically require extensive user testing prior to deployment, creating costly and time-consuming bottlenecks in industrial development pipelines, as evaluation depends on real users interacting with the system. Additionally, collecting conversational data introduces privacy concerns that hinder deployment in regulated settings (e.g., healthcare and financial area) . User simulators provide a promising solution to these challenges by enabling scalable, low-cost, and privacy-preserving evaluation and training of CRSs without requiring extensive interaction with real users. User simulators are automated agents that emulate human interaction patterns in recommendation dialogues by generating responses that approximate real user behavior. LLM based user simulators have shown its great potential in this LLM era by serving as synthetic user simulators that enable reproducible, cost-effective user simulation in recommender systems across diverse user populations and interaction contexts [10]–[14] . The existing LLM user simulators can be categorized into fine-tuned/training based simulators [15]–[17] and prompt based simulators [18]–[22] . In this paper, we focus on promptbased simulators, as they are cost-effective compared to training/fine-tuning based methods which require fine-t

FAQs

Why is user simulation important for conversational recommender systems?

User testing for conversational recommenders is costly and time-consuming, and collecting conversational data is constrained by privacy. User simulators enable scalable, low-cost, and privacy-preserving evaluation and training without extensive real-user interactions.

What limitations affect existing LLM-based user simulators?

They often show systematic positive bias with unrealistically high acceptance rates, data leakage where profile history is echoed verbatim, and constrained behavioral diversity such as popularity bias and temporal clustering that reduces realism.

How does the proposed framework improve LLM user simulators?

It automatically optimizes prompts for LLM-based user simulators to mitigate positive bias, prevent data leakage effects, and increase behavioral alignment with real human interaction patterns across diverse prompt settings.

Jul 16, 2026

Upload by Emma Wilson