[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-seo-513521-105":3,"detail-sidebar-cat-0-en-105":81,"doc-detail-513521-en":130},{"code":4,"msg":5,"data":6},0,"ok",{"site_id":7,"language":8,"slug":9,"title":10,"keywords":11,"description":12,"schema_data":13,"social_meta":74,"head_meta":76,"extra_data":78,"updated_unix":80},105,"en","from-passive-delegates-to-strategic-negotiators-reinforcing-social-reasoning-in-small-language-models-with-socialrl","FROM PASSIVE DELEGATES TO STRATEGIC NEGOTIATORS - REINFORCING SOCIAL REASONING IN SMALL LANGUAGE MODELS WITH SOCIALRL","","AI agents increasingly act for users as delegates, handling tasks like scheduling, offer comparison, and price haggling, yet they must navigate counterpart goals that can conflict with their principal’s interests. Helpful assistant behaviors can also create delegate failures, including unprompted disclosure of private information and premature concession. SOCIALRL is introduced as a training recipe for social reasoning, evaluated on a 4B model across six negotiation and coordination domains, with results showing strong in-domain performance, structured cross-domain transfer, and unified multi-domain utility gains.",{"@graph":14,"@context":73},[15,34,56],{"@type":16,"itemListElement":17},"BreadcrumbList",[18,23,27,31],{"item":19,"name":20,"@type":21,"position":22},"https://docshare.wps.com","Home","ListItem",1,{"item":24,"name":25,"@type":21,"position":26},"https://docshare.wps.com/document/","Document",2,{"item":28,"name":29,"@type":21,"position":30},"https://docshare.wps.com/document/research-report/","Research & Report",3,{"item":32,"name":10,"@type":21,"position":33},"https://docshare.wps.com/document/from-passive-delegates-to-strategic-negotiators-reinforcing-social-reasoning-in-small-language-models-with-socialrl/513521/",4,{"url":32,"name":10,"@type":35,"image":36,"author":41,"headline":10,"publisher":44,"fileFormat":47,"inLanguage":8,"description":12,"dateModified":48,"datePublished":49,"encodingFormat":47,"isAccessibleForFree":50,"interactionStatistic":51},"DigitalDocument",{"url":37,"@type":38,"width":39,"height":40},"https://docshare.wps.com/thumbnails/from-passive-delegates-to-strategic-negotiators-reinforcing-social-reasoning-in-small-language-models-with-socialrl/513521.png","ImageObject",300,407,{"name":42,"@type":43},"Blue Pony","Person",{"url":19,"name":45,"@type":46},"DocShare","Organization","application/pdf","2026-11-03","2026-10-06",true,{"@type":52,"interactionType":53,"userInteractionCount":55},"InteractionCounter",{"@type":54},"ViewAction",8,{"@type":57,"mainEntity":58},"FAQPage",[59,65,69],{"name":60,"@type":61,"acceptedAnswer":62},"What problem does SOCIALRL address in AI delegation scenarios?","Question",{"text":63,"@type":64},"Delegation becomes difficult when an agent must interact with counterparts whose goals conflict with its principal, and when “helpful” behaviors cause privacy leaks or premature concessions.","Answer",{"name":66,"@type":61,"acceptedAnswer":67},"How is SOCIALRL evaluated in the paper?",{"text":68,"@type":64},"The approach is applied to a 4B model and trained in-domain across six interaction domains, then evaluated across all six domains to test both per-domain performance and cross-domain transfer.",{"name":70,"@type":61,"acceptedAnswer":71},"What strategies and mechanisms improve performance across domains?",{"text":72,"@type":64},"The paper consolidates per-domain specialists using cascade RL and multi-teacher on-policy distillation, and it introduces a theory-of-mind scaffold where distilling ToM traces improves utility and generalization.","https://schema.org",{"og:url":32,"og:type":75,"og:title":10,"og:site_name":45,"og:description":12},"article",{"robots":77,"canonical":32},"index,follow",{"doc_id":79,"site_id":7},513521,1791394981,{"code":4,"msg":82,"data":83},"success",[84,88,92,96,101,106,111,114,119,122,126],{"id":22,"doc_module":4,"doc_module_name":25,"category_name":85,"show_sort_weight":86,"slug":87},"Story & Novel",90,"story-novel",{"id":26,"doc_module":4,"doc_module_name":25,"category_name":89,"show_sort_weight":90,"slug":91},"Literature",80,"literature",{"id":33,"doc_module":4,"doc_module_name":25,"category_name":93,"show_sort_weight":94,"slug":95},"Exam",70,"exam",{"id":97,"doc_module":4,"doc_module_name":25,"category_name":98,"show_sort_weight":99,"slug":100},5,"Comic",60,"comic",{"id":102,"doc_module":4,"doc_module_name":25,"category_name":103,"show_sort_weight":104,"slug":105},6,"Technology",50,"technology",{"id":107,"doc_module":4,"doc_module_name":25,"category_name":108,"show_sort_weight":109,"slug":110},7,"Healthcare",40,"healthcare",{"id":55,"doc_module":4,"doc_module_name":25,"category_name":29,"show_sort_weight":112,"slug":113},30,"research-report",{"id":115,"doc_module":4,"doc_module_name":25,"category_name":116,"show_sort_weight":117,"slug":118},9,"Religion & Spirituality",20,"religion-spirituality",{"id":117,"doc_module":4,"doc_module_name":25,"category_name":120,"show_sort_weight":117,"slug":121},"World Cup","world-cup",{"id":123,"doc_module":4,"doc_module_name":25,"category_name":124,"show_sort_weight":123,"slug":125},10,"Lifestyle","lifestyle",{"id":127,"doc_module":4,"doc_module_name":25,"category_name":128,"show_sort_weight":97,"slug":129},19,"General","general",{"code":4,"msg":82,"data":131},{"doc_id":79,"user_id":132,"nickname":42,"user_avatar":133,"doc_module":4,"category_id":55,"category_name":29,"doc_title":10,"doc_description":12,"doc_content":134,"file_id":135,"file_url":136,"file_type":137,"file_size":138,"view_count":55,"is_deleted":4,"is_public":22,"is_downloadable":22,"audit_status":22,"page_count":139,"language":140,"language_code":8,"site_id":7,"html_lang":8,"table_of_contents":141,"faqs":142,"seo_title":143,"seo_description":12,"update_tm":144,"read_time":145,"preview_page_count":33,"preview_urls":146},962090760266,"https://ap-avatar.wpscdn.com/davatar_276721f389ce27ea32af1340a28f341c","arXiv :2608 . 13787v1 [ cs .AI] 13 Aug 2026  \nFROM PASSIVE DELEGATES TO STRATEGIC NEGOTIATORS: REINFORCING SOCIAL REASONING IN SMALL LANGUAGE MODELS WITH SOCIALRL  \nWenyue HuaZachary Huang, Tyler Payne, Safoora Yousefi, Saleema Amershi, and Asli Celikyilmaz  \nMicrosoft Research, AI Frontiers  \nABSTRACT  \nAI agents increasingly act on their users’ behalf as representatives of their interests, handling tasks such as scheduling meetings, comparing offers, and haggling over prices. These principal-driven tasks routinely place the agent across from a counterpart, such as another user’s agent, a seller, ora recruiter, whose goals may conflict with those of its principal. Yet the dispositions that make an assistant pleasant can make it a poor delegate: a friendly and helpful frontier model may disclose its principal’s private information unprompted, and concede the principal’s position at the first sign of resistance. We present SOCIALRL, a general recipe that trains social reasoning directly, and apply it to a 4B model across six interaction domains: Deal-or-No-Deal, CaSiNo, Craigslist, Job Interview, Calendar, and Marketplace. Every domain is trained in-domain under the same recipe, and every trained policy is evaluated on all six domains. We find that (1) in-domain training reaches the frontier: on held-out scenarios the 4B matches or exceeds the GPT-5 family per domain, closing 73–122% of the baseline-to-frontier gap on the negotiation games, with the change visible at the trace level where 78% of buyer openings anchor below target versus 3% untrained; (2) cross-domain transfer follows game structure: structurally paired games lift each other, a broad multi-issue donor lifts nearly all domains, and structurally isolated games transfer nothing; (3) guided by this transfer structure, we propose two strategies to consolidate the per-domain specialists into a single unified 4B that achieves 0.627 average utility across all six environments, matching or exceeding GPT- 4.1 (0.625), GPT-5.1 (0.619), GPT-5.2 (0.613): cascade RL and multi-teacher on-policy distillation (OPD); (4) an explicit theory-of-mind (ToM) scaffold helps only through training: distilling the ToM trace, rather than actions alone, lifts utility on every environment and generalizes better across them, and of the two ToM skills, only next-action prediction predicts negotiation outcomes.  \n1 INTRODUCTION  \nAI agents are no longer only tools for completing tasks. A growing share of what they are asked to do carries the user’sstake: the agent acts on the user’s behalf, and the outcome bears directly on the user’s interests (Sun et al., 2025b; South et al., 2025) . Deployed systems can manage customer-service requests, coordinate meetings, purchase goods and book reservations, and help users compare homes or schedule property tours (Kovala, 2026; Google, 2026; OpenAI, 2025; Zillow, 2026) . These applications position an AI agent as a delegate: a system entrusted with a principal’s preferences and authorized to take consequential actions in open-ended environments.  \nDelegation becomes especially challenging when an agent interacts with another party whose objectives differ from those of its principal. A customer-service agent (Chaturvedi & Verma, 2023) may face a user seeking compensation, a purchasing agent may negotiate with a seller seeking a higher price (Haurum et al., 2024), and a scheduling agent (Zou et al., 2026) may need to reconcile participants with competing constraints. In such settings, merely completing the interaction is insufficient. A capable delegate must protect private information (Juneja et al., 2025), infer the counterpart’s incentives (Matta, 2026), determine when to concede or push back, and pursue an agreement that advances its principal’s interests (Sun et al., 2025a) . We refer to this collection of capabilities as social reasoning: reasoning about another actor’s latent preferences, incentives, and likely behavior in order to choose strategical","cbCaiuXzOvQlWx4e","https://ap.wps.com/l/cbCaiuXzOvQlWx4e","pdf",3334686,25,"English","# Abstract\n# 1 Introduction\n## Delegation as user-stakes task\n## Social reasoning in strategic interactions\n## Negotiation as a testbed for strategic behavior","[{\"question\":\"What problem does SOCIALRL address in AI delegation scenarios?\",\"answer\":\"Delegation becomes difficult when an agent must interact with counterparts whose goals conflict with its principal, and when “helpful” behaviors cause privacy leaks or premature concessions.\"},{\"question\":\"How is SOCIALRL evaluated in the paper?\",\"answer\":\"The approach is applied to a 4B model and trained in-domain across six interaction domains, then evaluated across all six domains to test both per-domain performance and cross-domain transfer.\"},{\"question\":\"What strategies and mechanisms improve performance across domains?\",\"answer\":\"The paper consolidates per-domain specialists using cascade RL and multi-teacher on-policy distillation, and it introduces a theory-of-mind scaffold where distilling ToM traces improves utility and generalization.\"}]","FROM PASSIVE DELEGATES TO STRATEGIC NEGOTIATORS - REINFORCING SOCIAL REASONING IN SMALL LANGUAGE MODELS WITH SOCIALRL | PDF",1791286573,63,[147,151,155,159],{"page_no":22,"url":148,"width":149,"height":150},"https://res-academy.cache.wpscdn.com/upload/kdocshare/2026-11-17/603c133c2ef8bc0e0870e49ed01ed0c4.webp",983,1272,{"page_no":26,"url":152,"width":153,"height":154},"https://res-academy.cache.wpscdn.com/upload/kdocshare/2026-11-17/75aa063b29a2fa07e7e99be11b5e41a9.webp",982,1270,{"page_no":30,"url":156,"width":157,"height":158},"https://res-academy.cache.wpscdn.com/upload/kdocshare/2026-11-17/3ac9cf84c55dc30fe7786cd997386575.webp",981,1269,{"page_no":33,"url":160,"width":161,"height":162},"https://res-academy.cache.wpscdn.com/upload/kdocshare/2026-11-17/3ef7d3dfcf166d3abe84773976097f88.webp",980,1268]