[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84894-en":3,"doc-seo-84894-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":13,"seo_description":14,"update_tm":28,"read_time":29},84894,8796095461610,"Oliver","https://ap-avatar.wpscdn.com/davatar_276721f389ce27ea32af1340a28f341c",8,"Research & Report","Pluralis v0.1: Towards a Multicultural, Multimodal, Multilingual Benchmark for AI Risk and Reliability","Current AI safety evaluation and benchmarking largely assume Western-centric defaults, framing alignment as a binary, culture-agnostic property. This approach can hide region-specific legal constraints, socio-linguistic details, and cultural taboos, exposing vision-language models in real-world global deployments. Pluralis v0.1 introduces a culture-first multimodal, multi-regional, multilingual dataset built for Asia-Pacific settings, plus Judge-Pluralis: an agreement-gated LLM-as-a-judge ensemble trained on a cultural taxonomy to surface locale-specific failures and contextual interaction errors.","arXiv :2607 .06 196v 1 [ cs .CL] 7 Jul 2026  \nPluralis v0 . 1: Towards a Multicultural, Multimodal, Multilingual Benchmark for AI Risk and Reliability  \nAlicia Parrish 1,* Rajat Shinde2,*  \nSanket Badhe3, Xinyi Bai 1, Sree Bhargavi Balija4, Hua-Rong Chu5, Emilio  \nFerrara6, Armstrong Foundjem7, Rajat Ghosh8, Aakash Gupta9, Xuanli He 10, Ong Chen Hui 11, Minji Jung23, Madhangi Karimanal 15, Faiza Khan Khattak 12, Boryoung Kim 13, Eugenia Kim 14, Liliya Lavitas3, Seok Min Lim 11, Victor Lu 16, Jim Moirangthem3, Dhivya Nagasubramanian26, Deepak Pandita27, Sita Rajagopal 11, Geetha Raju 15, Evgeniia Razumovskaia 1, Aravind Reddy 15, Federico Ricciuti 16, Nobin Sarwar 17, Sungpil Shin29, Sunayana Sitaram 18, Snehal Thorat3, Tharindu  \nCyril Weerasooriya 16,  \nJasmijn Bastings 1 Joachim Baumann 19 Kongtao Chen3 Murali Emani20 Mariya  \nHendriksen21 Jiho Jin22 Jun Seong Kim22 Younghoon Ko 13 Alicja Kwasniewska24 Minjae Lee23 Tom Wei-cyuan Lin Kashyap Ramanandula Manjusha25 Junho  \nMyung22 Junyeong Park22 Roma Patel 1 Shyam Ratan 18 Sudarsun Santhiappan 15 Priyanka Suresh 1 Tuesday32 Ksheeraj Sai Vepuri Laura Amortegui-Ordonez30,  \nClaire Dennis 14,  \nMinsuk Kahng23, Chris Knotz30,31, Alice Oh22, Balaraman Ravindran 15, Soojung Ryu 13,28, William Bartholomew 14, Hiwot Tesfaye 14, Lora Aroyo 1,  \n1 Google DeepMind, 2 University of Alabama in Huntsville, 3 Google, 4 University of Missouri Columbia, 5 Chunghwa Telecom Laboratories, 6 University of Southern California, Thomas Lord Department of Computer Science, 7 Polytechnique Montreal, 8 Nutanix, 9 ThinkEvolve Labs, 10 UCL, 11 Infocomm Media Development Authority, 12 Monark Health, 13 Seoul National University, 14 Microsoft, 15 Centre for Responsible AI (CeRAI), Wadhwani School of Data Science and AI (WSAI), Indian Institute of Technology Madras, 16 Independent,  \n17 University of Maryland, Baltimore County, 18 Microsoft Research India, 19 Stanford University, 20 Argonne National Laboratory, 21 University of Oxford, 22 KAIST, 23Yonsei University, 24 Amazon, 25 UIUC, 26 Independent Researcher, 27 Rochester Institute of  \nTechnology, 28 Xenoscube Inc. , 29 Korea AI Safety Institute (K-AISI), 30 MLCommons, 31 CommonGround, 32 Artifex Labs  \n* Lead author (equal contribution) †Core author ‡Project lead  \nJuly 2026  \nAbstract  \nCurrent AI safety evaluation and benchmarking frameworks predominantly rely on Western-centric defaults, treating model alignment as a binary, culture-agnostic property. This paradigm optimizes for global consensus but masks critical regional laws, socio-linguistic nuances, and cultural taboos, leaving Vision-Language Models (VLMs) vulnerable in global deployments. To address this, we introduce Pluralis v0.1-a novel multimodal, multi-regional, and multilingual dataset built from a culture-first perspective. Spanning 6,448 prompts across six Asia-Pacific countries (e.g. Bangladesh, India, Korea, Pakistan, Singapore and Taiwan) and eight languages in those countries (e.g. local English variant as well as one or two official languages), Pluralis diverges from prior work by natively sourcing localized safety hazards rather than adapting Western datasets to Asia-Pacific region. Crucially, it introducesa novel multimodal evaluation paradigm: user text (e.g., \"Should I gift this?\") and an image referring to \"this\" (e.g., a clock) -both innocuous in isolation, but synergistically triggering specific legal or cultural violations when combined. Pluralis disentangles universal safety violations from localized cultural appropriateness, establishing the latter as a first-class evaluation axis. To operationalize this at scale, we present Judge-Pluralis, an agreement-gated LLM-as-a-Judge ensemble trained on examples classified in an empirically derived cultural taxonomy. Observing frontier VLM behavior on a subset of the Pluralis surfaces recurring, locale-specific failure modes such as image misidentifications with downstream harm, missed item-context-locale interactions, and inadeq","cbCaihEEXDplYNB7","https://ap.wps.com/l/cbCaihEEXDplYNB7","pdf",6442184,2,1,31,"English","en",105,"# Abstract\n# Introduction","[{\"question\":\"What limitation of existing AI safety benchmarks does Pluralis v0.1 address?\",\"answer\":\"It addresses the Western-centric view that treats safety as binary and culture-agnostic, which can mask region-specific legal, socio-linguistic, and cultural constraints in real deployments.\"},{\"question\":\"How does Pluralis v0.1 differ in dataset design from prior work?\",\"answer\":\"Pluralis v0.1 is built from a culture-first perspective, sourcing localized safety hazards natively across six Asia-Pacific countries and eight languages rather than adapting Western datasets.\"},{\"question\":\"What is Judge-Pluralis and how is it used in evaluation?\",\"answer\":\"Judge-Pluralis is an agreement-gated LLM-as-a-judge ensemble trained on examples classified using an empirically derived cultural taxonomy to detect locale-specific multimodal failure modes.\"}]",1784199150,78,{"code":4,"msg":31,"data":32},"ok",{"site_id":25,"language":24,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"pluralis-v01-towards-a-multicultural-multimodal-multilingual-benchmark-for-ai-risk-and-reliability","",{"@graph":36,"@context":85},[37,53,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,47,50],{"item":41,"name":42,"@type":43,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":20},"https://docshare.wps.com/document/","Document",{"item":48,"name":12,"@type":43,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":43,"position":52},"https://docshare.wps.com/document/pluralis-v01-towards-a-multicultural-multimodal-multilingual-benchmark-for-ai-risk-and-reliability/84894/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":24,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":41,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-19","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What limitation of existing AI safety benchmarks does Pluralis v0.1 address?","Question",{"text":75,"@type":76},"It addresses the Western-centric view that treats safety as binary and culture-agnostic, which can mask region-specific legal, socio-linguistic, and cultural constraints in real deployments.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does Pluralis v0.1 differ in dataset design from prior work?",{"text":80,"@type":76},"Pluralis v0.1 is built from a culture-first perspective, sourcing localized safety hazards natively across six Asia-Pacific countries and eight languages rather than adapting Western datasets.",{"name":82,"@type":73,"acceptedAnswer":83},"What is Judge-Pluralis and how is it used in evaluation?",{"text":84,"@type":76},"Judge-Pluralis is an agreement-gated LLM-as-a-judge ensemble trained on examples classified using an empirically derived cultural taxonomy to detect locale-specific multimodal failure modes.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":21,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":20,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]