[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-117865-en":3,"doc-seo-117865-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},117865,34359740700684,"Finn","https://ap-avatar.wpscdn.com/avatar/1f400023980c374ae676?_k=1777273430885731487",8,"Research & Report","Machine Learning - Misinformation - and Citizen Science","Current methods for operationalizing misinformation in machine learning often fail to match the distinct success conditions found in other natural and social science models. Because misinformation is intrinsically value-laden and because citizens’ and social scientists’ concepts of misinformation evolve dynamically, construct legitimacy and construct validity of these models require evaluation through more democratic criteria than previously recognized. The work reframes how MMMs depend on stakeholder judgments and how those epistemic inputs shape model assessment.","Machine Learning, Misinformation, and Citizen Science 􀀃  \nAdrian K. Yee  \n(Lingnan University, Department of Philosophy, Hong Kong Catastrophic Risk  \nCentre)  \nForthcoming in European Journal for Philosophy of Science  \nAbstract  \nCurrent methods of operationalizing concepts of misinformation in machine learning are often problematic given idiosyncrasies in their success conditions compared to other models employed in the natural and social sciences. The intrinsic valueladenness of misinformation and the dynamic relationship between citizens' and social scientists' concepts of misinformation jointly suggest that both the construct legitimacy and the construct validity of these models needs to be assessed via more democratic criteria than has previously been recognized.  \nIntroduction  \nMachine learning models of misinformation (MMMs) that identify and censor misinformation are increasingly prevalent in private industry, government, and academic research, including computer science (Khan et al. 2021; Shu & Liu 2019), political science (Guess et al. 2019), engineering (Caled & Silva 2022), climate physics (Coan et al. 2021), medicine (Du et al. 2021), and especially information science (Nevo & Horne 2022; Gruppi et al. 2020) . This is so given that misinformation has spread faster and more widely than ever before due to the advent of the internet and social media platforms (Vosoughi et al. 2018) . MMMs have become sophisticated and established enough that there are textbooks on core techniques (Shu & Liu 2019) and are increasingly predictively powerful, performing well on standard performance criteria such as accuracy, precision, and recall (Mishra et al. 2022, 13; Khan et al. 2021, 8) . This suggests that there is preliminary evidence for the empirical adequacy of MMMs.  \nHowever, our understanding of misinformation remains inchoate given that there remains considerable debate as to how to de􀀌ne and operationalize misinformation and cognate concepts such as `fake news' (Habgood-Coote 2019; Pennycook & Rand 2021), persistent over reliance on epistemic elites' biased 􀀌rst-order judgments of what counts as  \n􀀃 Correspondence may be addressed to [adriankyle.yee@mail.utoronto.ca](adriankyle.yee@mail.utoronto.ca). I thank the following for constructive feedback on ideas in this paper: Brian Baigrie, Franz Huber, Michael Miller, Regina Rini, Denis Walsh, the Pittsburgh HPS fringe theory group, the York University moral psychology lab, and four anonymous reviewers. All errors and infelicities are mine alone.  \nmisinformation (Yee 2023), and the extent at which extant MMMs can capture violations of Gricean maxims in natural language processing (S􀀜e 2018) . This raises concerns about the initial construct legitimacy of the judgments of misinformation used in extant MMMsas well as the construct validity of the operationalizations employed.  \nThis paper discusses the epistemic work􀀍ow of MMMs, and connects this to a broader philosophical discussion in the literature on construct legitimacy and construct validity in the philosophy of science. Most MMMs typically rely upon supervised learning via the labeling of data by a diversity of stakeholders that include researchers, journalists, and average citizens recruited from services such as Amazon Mechanical Turk (Sorokin & Forsyth 2008) . I argue that this renders the construction of MMMs relevant to theories of citizen science in unexpected ways that impact MMMs' construct legitimacy and construct validity, and that this connection is a critically neglected aspect of MMMs. As I will show, extant MMMs often end up in practice overly privileging the judgments of epistemic elites who operationalize concepts of misinformation into these models. Given that misinformation is an intrinsically value-laden concept, whose operationalization necessarily concerns a diversity of stakeholders, MMMs require more stakeholder engagement than is currently the case. Furthermore, misinformation cannot be direct","cbCainXqWuXfnRNL","https://ap.wps.com/l/cbCainXqWuXfnRNL","pdf",259323,1,28,"English","en",105,"# Introduction\n## MMMs and the operationalization problem\n## Construct legitimacy vs construct validity\n## Citizen science implications for MMMs","[{\"question\":\"Why are current misinformation operationalizations in machine learning often problematic?\",\"answer\":\"They mismatch the idiosyncratic success conditions of misinformation models compared with other models used in the natural and social sciences.\"},{\"question\":\"How do value-ladenness and citizen–scientist differences affect evaluating MMMs?\",\"answer\":\"Misinformation carries intrinsic values, and citizens’ and social scientists’ concepts shift dynamically, so both construct legitimacy and construct validity need democratic assessment criteria.\"},{\"question\":\"What role does stakeholder engagement play in the design of MMMs?\",\"answer\":\"Because misinformation depends on projections of information quality and involves diverse stakeholders’ judgments, MMMs require more stakeholder engagement than current practice typically provides.\"}]","Machine Learning - Misinformation - and Citizen Science | PDF",1785680066,71,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"machine-learning-misinformation-and-citizen-science","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/machine-learning-misinformation-and-citizen-science/117865/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-02",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why are current misinformation operationalizations in machine learning often problematic?","Question",{"text":75,"@type":76},"They mismatch the idiosyncratic success conditions of misinformation models compared with other models used in the natural and social sciences.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How do value-ladenness and citizen–scientist differences affect evaluating MMMs?",{"text":80,"@type":76},"Misinformation carries intrinsic values, and citizens’ and social scientists’ concepts shift dynamically, so both construct legitimacy and construct validity need democratic assessment criteria.",{"name":82,"@type":73,"acceptedAnswer":83},"What role does stakeholder engagement play in the design of MMMs?",{"text":84,"@type":76},"Because misinformation depends on projections of information quality and involves diverse stakeholders’ judgments, MMMs require more stakeholder engagement than current practice typically provides.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]