[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"detail-sidebar-cat-0-en-105":3,"doc-seo-609312-105":59,"doc-detail-609312-en":130},{"code":4,"msg":5,"data":6},0,"success",[7,13,18,23,28,33,38,43,48,51,55],{"id":8,"doc_module":4,"doc_module_name":9,"category_name":10,"show_sort_weight":11,"slug":12},1,"Document","Story & Novel",90,"story-novel",{"id":14,"doc_module":4,"doc_module_name":9,"category_name":15,"show_sort_weight":16,"slug":17},2,"Literature",80,"literature",{"id":19,"doc_module":4,"doc_module_name":9,"category_name":20,"show_sort_weight":21,"slug":22},4,"Exam",70,"exam",{"id":24,"doc_module":4,"doc_module_name":9,"category_name":25,"show_sort_weight":26,"slug":27},5,"Comic",60,"comic",{"id":29,"doc_module":4,"doc_module_name":9,"category_name":30,"show_sort_weight":31,"slug":32},6,"Technology",50,"technology",{"id":34,"doc_module":4,"doc_module_name":9,"category_name":35,"show_sort_weight":36,"slug":37},7,"Healthcare",40,"healthcare",{"id":39,"doc_module":4,"doc_module_name":9,"category_name":40,"show_sort_weight":41,"slug":42},8,"Research & Report",30,"research-report",{"id":44,"doc_module":4,"doc_module_name":9,"category_name":45,"show_sort_weight":46,"slug":47},9,"Religion & Spirituality",20,"religion-spirituality",{"id":46,"doc_module":4,"doc_module_name":9,"category_name":49,"show_sort_weight":46,"slug":50},"World Cup","world-cup",{"id":52,"doc_module":4,"doc_module_name":9,"category_name":53,"show_sort_weight":52,"slug":54},10,"Lifestyle","lifestyle",{"id":56,"doc_module":4,"doc_module_name":9,"category_name":57,"show_sort_weight":24,"slug":58},19,"General","general",{"code":4,"msg":60,"data":61},"ok",{"site_id":62,"language":63,"slug":64,"title":65,"keywords":66,"description":67,"schema_data":68,"social_meta":123,"head_meta":125,"extra_data":127,"updated_unix":129},105,"en","validity-arguments-meet-artificial-intelligence-in-innovative-educational-assessment","Validity Arguments Meet Artificial Intelligence in Innovative Educational Assessment","","Discusses how validity arguments for educational assessment must adapt to AI-enabled scoring from conception through reporting. Emphasizes that, despite promises of efficiency and fairness, AI-based methods can complicate or amplify challenges around validity and bias in high-stakes contexts. Reviews impacts of Evidence-Centered Design on evidence generation and development, outlines ways to prevent embedded bias, and considers future directions to maximize AI benefits while minimizing negative effects on test takers and programs.",{"@graph":69,"@context":122},[70,84,105],{"@type":71,"itemListElement":72},"BreadcrumbList",[73,77,79,82],{"item":74,"name":75,"@type":76,"position":8},"https://docshare.wps.com","Home","ListItem",{"item":78,"name":9,"@type":76,"position":14},"https://docshare.wps.com/document/",{"item":80,"name":40,"@type":76,"position":81},"https://docshare.wps.com/document/research-report/",3,{"item":83,"name":65,"@type":76,"position":19},"https://docshare.wps.com/document/validity-arguments-meet-artificial-intelligence-in-innovative-educational-assessment/609312/",{"url":83,"name":65,"@type":85,"image":86,"author":91,"headline":65,"publisher":94,"fileFormat":97,"inLanguage":63,"description":67,"dateModified":98,"datePublished":99,"encodingFormat":97,"isAccessibleForFree":100,"interactionStatistic":101},"DigitalDocument",{"url":87,"@type":88,"width":89,"height":90},"https://docshare.wps.com/thumbnails/validity-arguments-meet-artificial-intelligence-in-innovative-educational-assessment/609312.png","ImageObject",300,407,{"name":92,"@type":93},"Franzy","Person",{"url":74,"name":95,"@type":96},"DocShare","Organization","application/pdf","2026-10-30","2026-10-18",true,{"@type":102,"interactionType":103,"userInteractionCount":29},"InteractionCounter",{"@type":104},"ViewAction",{"@type":106,"mainEntity":107},"FAQPage",[108,114,118],{"name":109,"@type":110,"acceptedAnswer":111},"Why do validity arguments need special attention when AI is used in educational assessment?","Question",{"text":112,"@type":113},"AI methods and tools can increase complexity, and the risks of validity and fairness challenges may not decrease and can even be amplified. Validity must still be supported, especially in high-stakes settings where bias matters.","Answer",{"name":115,"@type":110,"acceptedAnswer":116},"What elements underpin principled assessment design and development?",{"text":117,"@type":113},"Principled approaches include clearly defined assessment targets, intended score interpretations and uses, a model of cognition/learning/performance, aligned measurement and reporting scales, and assessment activity design aligned to the targets.",{"name":119,"@type":110,"acceptedAnswer":120},"How can NLP-based scoring require more rigorous validity evidence?",{"text":121,"@type":113},"When NLP is used to score text-based assessments, validity evidence should show that algorithms understand the produced words/text at relevant levels, recognize supporting reasoning, and evaluate whether that reasoning is relevant and appropriate.","https://schema.org",{"og:url":83,"og:type":124,"og:title":65,"og:site_name":95,"og:description":67},"article",{"robots":126,"canonical":83},"index,follow",{"doc_id":128,"site_id":62},609312,1792519142,{"code":4,"msg":5,"data":131},{"doc_id":128,"user_id":132,"nickname":92,"user_avatar":133,"doc_module":4,"category_id":39,"category_name":40,"doc_title":65,"doc_description":67,"doc_content":134,"file_id":135,"file_url":136,"file_type":137,"file_size":138,"view_count":29,"is_deleted":4,"is_public":8,"is_downloadable":8,"audit_status":8,"page_count":24,"language":139,"language_code":63,"site_id":62,"html_lang":63,"table_of_contents":140,"faqs":141,"seo_title":142,"seo_description":67,"update_tm":143,"read_time":144},2336478945519,"https://ap-avatar.wpscdn.com/davatar_085a072bc5b1113ac321206ff7593b45","DOI: 10.1111/jedm.12331 Journal of Educational Measurement  \nFall 2022, Vol. 59, No. 3, pp. 267–271  \nValidity Arguments Meet Artificial Intelligence in Innovative Educational Assessment  \nDavid W. Dorsey and Hillary R. Michaels  \nHuman Resources Research Organization (HumRRO)  \nWe have dramatically advanced our ability to create rich, complex, and effective assessments across a range of uses through technology advancement. Artificial Intelligence (AI) enabled assessments represent one such area of advancement—one that has captured our collective interest and imagination. Scientists and practitioners within the domains of organizational and workforce assessment have in creasingly used AI in assessment, and its use is now becoming more common in education. While these types of solutions offer their users the promise of efficiency, effectiveness, anda “wow factor,” users need to maintain high standards for validity and fairness in high stakes settings. Due to the complexity of some AI methods and tools, this requirement for adherence to standards may challenge our traditional ap proaches to building validity and fairness arguments. In this edition, we review what these challenges may look like as validity arguments meet AI in educational assess ment domains. We specifically explore how AI impacts Evidence-Centered Design (ECD) and development from assessment concept and coding to scoring and report ing. We also present information on ways to ensure that bias is not built into these systems. Lastly, we discuss future horizons, many that are almost here, for maximiz ing what AI offers while minimizing negative effects on test takers and programs.  \nIntroduction  \nIn this special edition, we discuss how modern validity arguments must meet emerging forms of Artificial Intelligence (AI) enabled assessment, from conception through reporting. Advances in technology have increased our ability to create rich, complex, and effective assessments across a range of uses. Correspondingly, our approaches to gathering and presenting validity arguments have also evolved in measurement and psychometrics, in a manner that unifies assessment design, interpretation, use, and validation (Kane, 2013; Mislevy, 2018; Sireci, 2013) . Such approaches are critical, as concerns for phenomena such as bias are not always reduced in AI enabled scoring but can in fact be amplified (Arthurs & Alvero, 2020) .  \nPrincipled approaches to assessment design, development, and implementation feature several foundational elements, including (a) clearly defined assessment targets,(b) a statement of intended score interpretations and uses,(c) a model of cognition, learning, or performance,(d) aligned measurement models and reporting scales, and (e) manipulation of assessment activities to align with assessment targets (Ferrara et al., 2016, p. 41) . These elements require special consideration when employing AI-based technology.  \nAs outlined by Robert Brennan (2010), principled design approaches are generally consistent with Kane’s (2006) validity argument approach; however, validity arguments are more extensive and focus on consequences and generalizability of  \n© 2022 by the National Council on Measurement in Education. 267  \nDorsey and Michaels  \nresults. Consequently, the focus within validity arguments needs to be on test score interpretations and uses. Therefore, it is incumbent upon test developers and users to understand how assessments are scored—something that potentially becomes more complicated when using AI enabled approaches.  \nSuch principled approaches to validity help us not only with familiar forms oftesting and assessment, but their greatest value may be found in reasoning through uses of emerging digital technologies, evolving measurement models, new statistical and data science methods, and advances in areas of social and behavioral science (e.g., cognitive, social) . Moreover, as we seek to measure more multifaceted constructs, using more complex assessment","cbCaij2xEN3jeT4n","https://ap.wps.com/l/cbCaij2xEN3jeT4n","pdf",185253,"English","# Introduction\n## Validity arguments in AI-enabled assessment\n## Principles for assessment design\n## Evidence-centered design and evidence generation","[{\"question\":\"Why do validity arguments need special attention when AI is used in educational assessment?\",\"answer\":\"AI methods and tools can increase complexity, and the risks of validity and fairness challenges may not decrease and can even be amplified. Validity must still be supported, especially in high-stakes settings where bias matters.\"},{\"question\":\"What elements underpin principled assessment design and development?\",\"answer\":\"Principled approaches include clearly defined assessment targets, intended score interpretations and uses, a model of cognition/learning/performance, aligned measurement and reporting scales, and assessment activity design aligned to the targets.\"},{\"question\":\"How can NLP-based scoring require more rigorous validity evidence?\",\"answer\":\"When NLP is used to score text-based assessments, validity evidence should show that algorithms understand the produced words/text at relevant levels, recognize supporting reasoning, and evaluate whether that reasoning is relevant and appropriate.\"}]","Validity Arguments Meet Artificial Intelligence in Innovative Educational Assessment | PDF",1792358541,13]