[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-85064-en":3,"doc-seo-85064-105":29,"detail-sidebar-cat-0-en-105":90},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},85064,1099514067438,"River Wang","https://ap-avatar.wpscdn.com/avatar/100002539ee87300030?x-image-process=image/resize,m_fixed,w_180,h_180&k=1780474512215547542",7,"Healthcare","MentalHospital: A Virtual Environment for Evaluating Psychiatric Clinical Encounters","Large language models (LLMs) perform well on individual psychiatric tasks such as dialogue, diagnosis, and treatment planning, but existing benchmarks seldom recreate complete psychiatric clinical encounters. MentalHospital is introduced as a virtual evaluation environment that implements the S.O.A.P. workflow: Subjective Interviewing, Objective Examination, Diagnostic Assessment, and Treatment Planning. It uses skill-augmented standardized patients built from 1,193 de-identified EHR cases across major ICD-11 categories and 76 disorders. Dual-track scoring combines EHR-grounded objective reference comparison with clinician-aligned subjective process evaluation. MentalEval, five domain evaluators trained with rubric-grounded SFT and expert-guided DPO, supports scalable specialist judgments. Clinician surveys indicate high clinical fidelity (3.88/5) and strong expert alignment for evaluators (QWK 0.944), while benchmarking reveals a gap where LLMs trail clinicians by 37.28 points, with mental status assessment as a key bottleneck.","arXiv :2607 .08257v 1 [ cs .AI] 9 Jul 2026  \nMentalHospital: A Virtual Environment for Evaluating Psychiatric Clinical Encounters  \nYuming Yang1 , Xiao Sun1 , Yuanwei Zou1 , Zhengxiao Wu1 , Yun Chen2 Jiang Zhong1,†, Haoyang Zeng1 , Jingwang Huang1 , Kaiwen Wei1,†  \n1 School of Computer Science, Chongqing University, Chongqing, China  \n2 School of Computer Science, Hunan University, Changsha, China †Corresponding authors.  \n[ymyang@cqu.edu.cn](ymyang@cqu.edu.cn), [zhongjiang@cqu.edu.cn](zhongjiang@cqu.edu.cn), [weikaiwen@cqu.edu.cn](weikaiwen@cqu.edu.cn)  \nAbstract  \nLarge language models (LLMs) have shown strong performance on isolated psychiatric tasks, including dialogue, diagnosis, and treatment planning, yet existing benchmarks rarely simulate complete psychiatric clinical encounters. We introduce MentalHospital, a virtual evaluation environment for LLM-based psychiatric clinical encounters. MentalHospital instantiates the Subjective Interviewing, Objective Examination, Diagnostic Assessment, and Treatment Planning (S.O.A.P.) workflow, using skill-augmented standardized patients constructed from 1,193 de-identified psychiatric electronic health record (EHR) cases spanning all major ICD-11 categories and 76 disorders. Each encounter is assessed through a dualtrack protocol that combines objective comparison against EHR-derived references with subjective assessment of clinical process quality. To scale specialist judgment, we develop MentalEval, five domain-specific evaluators covering communication empathy, interviewing professionalism, clinical-note quality, diagnostic rigor, and treatment appropriateness, trained with rubric-grounded SFT and expert-guided DPO. Survey responses from 22 clinicians support MentalHospital’s clinical fidelity (3.88/5), while MentalEval achieves strong expert alignment with an average QWK of 0.944 . Benchmarking shows that even the strongest LLM trails clinicians by 37.28 percentage points in objective psychiatric competence, with mental status assessment as a key bottleneck.  \n1 Introduction  \nLarge language models (LLMs) have made substantial progress in psychiatry [1, 2] . Existing studies have reported strong performance on key clinical components, including dialogue [3–6], medical record understanding [7], diagnostic reasoning [8], and treatment planning [9] . However, success on isolated clinical components remains insufficient to reflect the integrated capability required in real-world clinical encounters.  \nRecent studies have increasingly recognized the limitations of isolated evaluation [10, 11] . Accordingly, several works have begun exploring virtual medical environments for more comprehensive clinical simulation and assessment [12–15] . Yet, as summarized in Table 1, these environments still exhibit three limitations: (1) Encounter Incompleteness. Existing environments still center on doctor–patient communication, rather than complete encounters across multiple clinical scenarios and tasks. (2) Data Inauthenticity. The environmental evidence and reference targets are often derived from synthetic or web-based data, limiting clinical fidelity. (3) Patient Simplification. Virtual patients are often reactive symptom carriers, rather than faithful and distinctive psychiatric individuals.  \nPreprint.  \nTable 1: Comparison of psychiatric benchmarks. Inter., Exam., Diag., and Treat. denote Interview, Examination, Diagnosis, and Treatment Planning, respectively. MentalHospital provides EHRgrounded full-process simulation, multi-perspective evaluation, and specialized evaluators.  \n\n| Benchmark | Data |  | Simulation |  | Evaluation |  |  |\n| --- | --- | --- | --- | --- | --- | --- | --- |\n|  | Data Source | Psychiatry | Scope | Interaction | Method | Reference | Evaluator |\n| HealthBench [3] | Physician-authored | ✗ | General Healthcare | ✗ | Rubric | Physician | ✗ |\n| MedDialogRubrics [17] | Synthetic | ✗ | Consultation / Diagnosis | ✗ | Rubric | ✗ | ✗ |\n| CPsyCoun [4] | Web | ✓ | Consulta","cbCaingQBqAf23Mj","https://ap.wps.com/l/cbCaingQBqAf23Mj","pdf",2827024,1,45,"English","en",105,"# Abstract\n# Introduction\n## Limitations of Existing Benchmarks\n## MentalHospital and S.O.A.P. Workflow\n## Evaluation Protocol and MentalEval","[{\"question\":\"What problem does MentalHospital address in psychiatric LLM evaluation?\",\"answer\":\"Existing benchmarks often evaluate isolated psychiatric components rather than end-to-end clinical encounters. MentalHospital creates a full-process virtual environment that simulates complete encounters using the S.O.A.P. workflow.\"},{\"question\":\"How are virtual patients and clinical scenarios constructed in MentalHospital?\",\"answer\":\"MentalHospital builds skill-augmented standardized patients from 1,193 de-identified psychiatric EHR cases, spanning major ICD-11 categories and 76 disorder diagnoses, with individualized symptom presentation and dynamic recall.\"},{\"question\":\"How is model performance evaluated and scaled in MentalHospital?\",\"answer\":\"Each encounter is assessed via dual-track scoring: objective comparison to EHR-derived references and subjective evaluation of clinical process quality. To scale specialist judgment, MentalEval provides five domain-specific evaluators trained with rubric-grounded SFT and expert-guided DPO.\"}]",1784200747,113,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":85,"head_meta":87,"extra_data":89,"updated_unix":27},"mentalhospital-a-virtual-environment-for-evaluating-psychiatric-clinical-encounters","",{"@graph":35,"@context":84},[36,53,67],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/healthcare/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/mentalhospital-a-virtual-environment-for-evaluating-psychiatric-clinical-encounters/85064/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":61,"encodingFormat":60,"isAccessibleForFree":62,"interactionStatistic":63},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-16",true,{"@type":64,"interactionType":65,"userInteractionCount":4},"InteractionCounter",{"@type":66},"ViewAction",{"@type":68,"mainEntity":69},"FAQPage",[70,76,80],{"name":71,"@type":72,"acceptedAnswer":73},"What problem does MentalHospital address in psychiatric LLM evaluation?","Question",{"text":74,"@type":75},"Existing benchmarks often evaluate isolated psychiatric components rather than end-to-end clinical encounters. MentalHospital creates a full-process virtual environment that simulates complete encounters using the S.O.A.P. workflow.","Answer",{"name":77,"@type":72,"acceptedAnswer":78},"How are virtual patients and clinical scenarios constructed in MentalHospital?",{"text":79,"@type":75},"MentalHospital builds skill-augmented standardized patients from 1,193 de-identified psychiatric EHR cases, spanning major ICD-11 categories and 76 disorder diagnoses, with individualized symptom presentation and dynamic recall.",{"name":81,"@type":72,"acceptedAnswer":82},"How is model performance evaluated and scaled in MentalHospital?",{"text":83,"@type":75},"Each encounter is assessed via dual-track scoring: objective comparison to EHR-derived references and subjective evaluation of clinical process quality. To scale specialist judgment, MentalEval provides five domain-specific evaluators trained with rubric-grounded SFT and expert-guided DPO.","https://schema.org",{"og:url":51,"og:type":86,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":88,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":91},[92,96,100,104,109,114,117,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":93,"show_sort_weight":94,"slug":95},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":97,"show_sort_weight":98,"slug":99},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":101,"show_sort_weight":102,"slug":103},"Exam",70,"exam",{"id":105,"doc_module":4,"doc_module_name":45,"category_name":106,"show_sort_weight":107,"slug":108},5,"Comic",60,"comic",{"id":110,"doc_module":4,"doc_module_name":45,"category_name":111,"show_sort_weight":112,"slug":113},6,"Technology",50,"technology",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":115,"slug":116},40,"healthcare",{"id":118,"doc_module":4,"doc_module_name":45,"category_name":119,"show_sort_weight":120,"slug":121},8,"Research & Report",30,"research-report",{"id":123,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":45,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":45,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":45,"category_name":136,"show_sort_weight":105,"slug":137},19,"General","general"]