[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-82933-en":3,"doc-seo-82933-105":29,"detail-sidebar-cat-0-en-105":95},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},82933,8796095461610,"Oliver","https://ap-avatar.wpscdn.com/davatar_276721f389ce27ea32af1340a28f341c",8,"Research & Report","PiSAs Benchmarking Contextual Integrity in Multi-User Agentic Systems","PiSAs (Privacy in Shared Agentic systems) benchmarks unintentional privacy leakage as LLM agents move from single-user assistants to shared organizational infrastructure. Existing contextual integrity (CI) benchmarks largely miss cross-user “data spillage” caused by inter-agent messages, shared memory, and agent outputs. PiSAs adds dual CI annotations covering task-context appropriateness and per-user visibility. Results show improved CI compliance from better design, but persistent bottlenecks from unreliable LLM judgment when filtering inappropriate content and restricting transmission to authorized users.","PiSAs: Benchmarking Contextual Integrity in Multi-User Agentic Systems  \nShubham Gupta1,2,3,∗, Nazanin Mohammadi Sepahvand1,2,4,∗, Abhinav Kumar5,  \nCem Subakan2,3, Spandana Gella1,4, Pierre-André Noël1,  \nPerouz Taslakian1,2,4, Eugene Bagdasarian5, Valentina Zantedeschi1,3  \n1ServiceNow AI Research 2Mila – Quebec AI Institute 3Université Laval  \n4McGill University 5University of Massachusetts Amherst ∗Equal contribution  \narXiv :2607 .053 18v 1 [ cs .MA] 6 Jul 2026  \nAbstract  \nAs LLM agents evolve from single-user assistants into shared organizational infrastructure, new privacy risks emerge: inappropriate information may not only be exposed through outputs for external recipients, but also internally across users through inter-agent messages, shared memory and agents. These data spillage risks are not captured by existing privacy benchmarks grounded in contextual integrity (CI) as they focus primarily on either single-user settings or interactions between independently owned agents. We introduce PiSAs (Privacy in Shared Agentic systems), a benchmark for assessing unintentional leaks with dual CI annotations: whether an information is appropriate for the task, and which users may legitimately access it. This enables direct measurement of cross-user spillage across agentic system components and interfaces, such as outputs, inter-agent communication, and memory. PiSAsis system-agnostic and supports evaluation across different agent topologies and memory regimes.  \nWe find that, although system design improves CI compliance, results are bottlenecked by incorrect LLM judgment calls: even state-of-the-art models fail to reliably filter inappropriate content or restrict transmission to authorized users. Our findings underscore the need for privacy-preserving strategies, beyond those studied in this work.  \n1 Introduction  \nLarge language model (LLM) agents are currently deployed to complete tasks on behalf of individual users. Extending their deployment to organizational settings, where agents coordinate across users rather than operating in isolation, has the potential to tap into collective intelligence for effectively automatizing tasks such as scheduling, resource allocation, performance reviews, and knowledge sharing. It also introduces considerable privacy challenges. A multi-user agentic system inevitably handles information that is appropriate in one context but not in another, creating opportunities for it to surface where it does not  \nbelong or spill to the wrong user. For instance, a team member’s medical appointment may be appropriately shared with HR but should not be surfaced when a colleague ask their availability for a meeting.  \nThe privacy risks introduced by a multi-user system are qualitatively different from those studied in singleusers settings (Mireshghallah et al., 2025; Bagdasarian et al., 2024; Shao et al., 2024; Zharmagambetovet al., 2025; Yagoubi et al., 2026). When one or multiple agents solve a task for one user, the concern is whether the output reveals sensitive information to the external recipient. When agents collaborate on behalf of multiple users, the risks multiply: sensitive attributes about one employee may propagate through inter-agent communication and become accessible to other employees who should not have access; personal data may accumulate in a shared memory store readable by all users regardless of organizational access rights; the final output may incorporate inappropriate information about individuals who never consented to have their information reach the recipient. Crucially, these failures are unintentional, i.e. they might arise from the ordinary mechanics of collaboration, and not from adversarial behaviors.  \nThese are violations of contextual integrity (CI): the principle that information flows appropriately when they match the norms ofthe social context in which the information was originally shared (Nissenbaum, 2004). Under CI, privacy is not a property of data in isolati","cbCail13MvRQjsPU","https://ap.wps.com/l/cbCail13MvRQjsPU","pdf",1154695,1,39,"English","en",105,"# Abstract\n# 1 Introduction\n## Contextual integrity and privacy risks in shared agentic systems\n## PiSAs benchmark design and dual CI annotations\n## System-agnostic evaluation across topologies and memory regimes","[{\"question\":\"What problem does PiSAs target in multi-user agentic systems?\",\"answer\":\"PiSAs targets unintentional privacy leaks where information appropriate in one context is exposed to the wrong users through agent outputs, inter-agent communication, or shared memory.\"},{\"question\":\"How does PiSAs apply contextual integrity (CI) in its evaluation?\",\"answer\":\"PiSAs grounds annotations in CI by labeling whether an attribute is appropriate for the task-context and specifying which users are legitimately allowed to view the attribute.\"},{\"question\":\"What are the two main failure modes PiSAs measures?\",\"answer\":\"It measures inappropriate disclosure during task information gathering and data spillage via agent-to-agent messages or shared memory access across users.\"},{\"question\":\"Why do results still underperform even when system design improves CI compliance?\",\"answer\":\"Findings indicate a bottleneck from incorrect LLM judgment calls, where state-of-the-art models cannot reliably filter inappropriate content or restrict transmission strictly to authorized users.\"}]",1784184086,98,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":90,"head_meta":92,"extra_data":94,"updated_unix":27},"pisas-benchmarking-contextual-integrity-in-multi-user-agentic-systems","",{"@graph":35,"@context":89},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/pisas-benchmarking-contextual-integrity-in-multi-user-agentic-systems/82933/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81,85],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does PiSAs target in multi-user agentic systems?","Question",{"text":75,"@type":76},"PiSAs targets unintentional privacy leaks where information appropriate in one context is exposed to the wrong users through agent outputs, inter-agent communication, or shared memory.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does PiSAs apply contextual integrity (CI) in its evaluation?",{"text":80,"@type":76},"PiSAs grounds annotations in CI by labeling whether an attribute is appropriate for the task-context and specifying which users are legitimately allowed to view the attribute.",{"name":82,"@type":73,"acceptedAnswer":83},"What are the two main failure modes PiSAs measures?",{"text":84,"@type":76},"It measures inappropriate disclosure during task information gathering and data spillage via agent-to-agent messages or shared memory access across users.",{"name":86,"@type":73,"acceptedAnswer":87},"Why do results still underperform even when system design improves CI compliance?",{"text":88,"@type":76},"Findings indicate a bottleneck from incorrect LLM judgment calls, where state-of-the-art models cannot reliably filter inappropriate content or restrict transmission strictly to authorized users.","https://schema.org",{"og:url":51,"og:type":91,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":93,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":96},[97,101,105,109,114,119,124,127,132,135,139],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":106,"show_sort_weight":107,"slug":108},"Exam",70,"exam",{"id":110,"doc_module":4,"doc_module_name":45,"category_name":111,"show_sort_weight":112,"slug":113},5,"Comic",60,"comic",{"id":115,"doc_module":4,"doc_module_name":45,"category_name":116,"show_sort_weight":117,"slug":118},6,"Technology",50,"technology",{"id":120,"doc_module":4,"doc_module_name":45,"category_name":121,"show_sort_weight":122,"slug":123},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":125,"slug":126},30,"research-report",{"id":128,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":130,"slug":131},9,"Religion & Spirituality",20,"religion-spirituality",{"id":130,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":130,"slug":134},"World Cup","world-cup",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":136,"slug":138},10,"Lifestyle","lifestyle",{"id":140,"doc_module":4,"doc_module_name":45,"category_name":141,"show_sort_weight":110,"slug":142},19,"General","general"]