[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-120990-en":3,"doc-seo-120990-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},120990,4398048950312,"Violet","https://ap-avatar.wpscdn.com/avatar/400002538284de19e3c?_k=1778320343897328908",8,"Research & Report","Measuring Machine Learning Harms from Stereotypes - Requires Understanding Who Is Harmed by Which Errors in What Ways","Machine learning research increasingly documents how models can propagate harmful stereotypes, yet limited work centers on the lived psychological experiences of people encountering such stereotypes. This study uses a gender-stereotype case in image search to examine reactions to model errors. Surveys show errors vary in whether they reflect stereotypes and in how harmful they are. Experiments randomly assign stereotype-reinforcing, stereotype-violating, or neutral errors, revealing experiential harm patterns that differ by gender and error type.","Measuring Machine Learning Harms from Stereotypes Requires Understanding Who Is Harmed by Which Errors in What Ways  \nAngelina Wang  \nStanford University, Cornell Tech USA [angelina.wang@cornell.edu](angelina.wang@cornell.edu)  \nXuechunzi Bai  \nUniversity of Chicago USA  \narXiv :2402 .04420v2 [ cs .CY] 24 May 2025  \nSolon Barocas  \nMicrosoft Research USA  \nSu Lin Blodgett  \nMicrosoft Research USA  \nAbstract  \nDespite a proliferation of research on the ways that machine learning models can propagate harmful stereotypes, very little of this work is grounded in the psychological experiences of people exposed to such stereotypes. We use a case study of gender stereotypes in image search to examine how people react to machine learning errors. First, we use surveys to show that not all machine learning errors reflect stereotypes nor are equally harmful. Then, in experimental studies we randomly expose participants to stereotypereinforcing, -violating, and-neutral machine learning errors. We find stereotype-reinforcing errors induce more experiential harm, while having minimal impact on participants’ cognitive beliefs, attitudes, or behaviors. This experiential harm impacts participants who are women more than those who are men. However, certain stereotype-violating errors are more experientially harmful for men, potentially due to perceived threats to masculinity. We conclude by proposing a more nuanced perspective on the harms of machine learning errors—one that depends on who is experiencing what harm and why.  \nCCS Concepts  \n• Social and professional topics → Gender; • Computing methodologies → Object recognition.  \nKeywords  \nstereotypes, machine learning fairness, harms from biases  \nACM Reference Format:  \nAngelina Wang, Xuechunzi Bai, Solon Barocas, and Su Lin Blodgett. 2025. Measuring Machine Learning Harms from Stereotypes Requires Understanding Who Is Harmed by Which Errors in What Ways. In The 2025 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’25), June 23–26, 2025, Athens, Greece. ACM, New York, NY, USA, 17 pages. [https://doi.org/10.1145/3715275.3732046](https://doi.org/10.1145/3715275.3732046)  \n1 Introduction  \nOver the past decade, researchers have demonstrated that machine learning models run the risk of learning stereotypical associations.  \nPlease use nonacm option or ACM Engage class to enable CC licenses  This work is licensed under a Creative Commons Attribution 4 .0 International License.  \nFAccT ’25, June 23–26, 2025, Athens, Greece © 2025 Copyright held by the owner/author(s) .  \nACM ISBN 979-8-4007-1482-5/2025/06  \n[https://doi.org/10.1145/3715275.3732046](https://doi.org/10.1145/3715275.3732046)  \nFor example, natural language processing models have been shown to associate women with homemakers and men with programmers [11], while computer vision models have been shown to associate women with shopping and men with driving [118] . These associations can cause machine learning models to make systematic errors, mistakenly reporting, for instance, that female doctors are nursesand that male nurses are doctors [94] . A rich literature has developed offering many more such examples, spurring calls to address the risk that machine learning models might make mistakes that perpetuate harmful stereotypes.  \nUnfortunately, portions of this literature have suffered from three main limitations that impede effective intervention. First, while some research has asked crowd workers to annotate when errors invoke stereotypes or has drawn on pre-existing inventories of stereotypes [9, 11, 15, 16, 98, 106], machine learning researchers often rely on their own moral intuitions to determine which categories of associations to investigate (e.g., associations between gender and occupation) and to judge which specific associations (e.g., the association between women and nursing) are stereotypes. As a result, certain categories of associations that broader populations might perceive as stereotypical have n","cbCaiteclcVNGtJQ","https://ap.wps.com/l/cbCaiteclcVNGtJQ","pdf",5277128,1,17,"English","en",105,"# Introduction\n## Limitations in existing stereotype-harm research\n## Stereotype reinforcement vs violation\n## Study overview","[{\"question\":\"Why does the paper argue that existing stereotype-harm research is limited?\",\"answer\":\"It highlights reliance on researchers’ moral intuitions to decide which associations are stereotypes and the tendency to treat all errors as equally harmful regardless of whether they reinforce or violate stereotypes.\"},{\"question\":\"How do the authors test how people react to different machine learning errors?\",\"answer\":\"They run surveys and experiments in image search contexts, asking participants to evaluate whether errors relate to stereotypes and then randomly exposing participants to stereotype-reinforcing, stereotype-violating, or neutral errors.\"},{\"question\":\"What differences does the study find between stereotype-reinforcing and stereotype-violating errors?\",\"answer\":\"Stereotype-reinforcing errors produce more experiential harm while having minimal effects on cognitive beliefs, attitudes, or behaviors; stereotype-violating errors show more experiential harm for men, potentially linked to perceived threats to masculinity.\"}]","Measuring Machine Learning Harms from Stereotypes - Requires Understanding Who Is Harmed by Which Errors in What Ways | PDF",1785733203,43,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"measuring-machine-learning-harms-from-stereotypes-requires-understanding-who-is-harmed-by-which-errors-in-what-ways","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/measuring-machine-learning-harms-from-stereotypes-requires-understanding-who-is-harmed-by-which-errors-in-what-ways/120990/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-03",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why does the paper argue that existing stereotype-harm research is limited?","Question",{"text":75,"@type":76},"It highlights reliance on researchers’ moral intuitions to decide which associations are stereotypes and the tendency to treat all errors as equally harmful regardless of whether they reinforce or violate stereotypes.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How do the authors test how people react to different machine learning errors?",{"text":80,"@type":76},"They run surveys and experiments in image search contexts, asking participants to evaluate whether errors relate to stereotypes and then randomly exposing participants to stereotype-reinforcing, stereotype-violating, or neutral errors.",{"name":82,"@type":73,"acceptedAnswer":83},"What differences does the study find between stereotype-reinforcing and stereotype-violating errors?",{"text":84,"@type":76},"Stereotype-reinforcing errors produce more experiential harm while having minimal effects on cognitive beliefs, attitudes, or behaviors; stereotype-violating errors show more experiential harm for men, potentially linked to perceived threats to masculinity.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]