[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-157994-en":3,"doc-seo-157994-105":31,"detail-sidebar-cat-0-en-105":100},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":28,"seo_description":14,"update_tm":29,"read_time":30},157994,4398048950312,"Violet","https://ap-avatar.wpscdn.com/avatar/400002538284de19e3c?_k=1778320343897328908",4,"Exam","Improving Test Validity and Accessibility with Digital-First Assessments","Digital-first high-stakes assessments drive a rethink of validity standards by reshaping user experience and assessment vocabulary. The paper examines how digital-first features—especially personalization, improved test design, and accessibility—can reduce construct-irrelevant variation while strengthening validity evidence. Using the Duolingo English Language Test as an example, it analyzes implications for fairness and standardisation, the opportunities created by CAT, and what computer-based testing changes for accessibility and inclusive assessment. The conclusion argues that these digital characteristics set new expectations for good assessment design.","Improving Test Validity and Accessibility with Digital-First Assessments  \nNaomi Care & Bryan Maddox  \nAssessment MicroAnalytics Ltd  \n[Naomi@microanalytics.co.uk](Naomi@microanalytics.co.uk) • [Bryan@microanalytics.co.uk](Bryan@microanalytics.co.uk)  \nAbstract  \nDigital-first high stakes assessments invite us to rethink the standards, user experience, and vocabulary of assessment validity. We explore the disruptive potential of digital-first high stakes assessments to set higher standards in test performance and validity. In this paper we describe the new lexicon that digital-first assessments have introduced into high stakes digital assessments such as personalisation, user experience, and accessibility. The features of digital high stakes assessments have the potential to reduce sources of construct-irrelevant variation and improve test validity. In conclusion we argue that the distinctive features of digital high stakes assessment challenge our understanding of good assessment design setting new standards for assessment performance and validity.  \nKeywords  \nHigh-stakes assessment, digital-first assessment, user experience, validity, inclusive assessment.  \nAbout the authors  \nNaomi Care has an academic background in education, anthropology, and psychology. As an Analyst with Assessment MicroAnalytics, she leads work in the area of digital ethnography, and on neurodiversity, disability and chronic illness in assessment.  \nBryan Maddox is Executive Director of Assessment MicroAnalytics, Professor in Educational Assessment at the University of East Anglia, and visiting professor at the Centre for Educational Measurement at the University of Oslo. Bryan has conducted assessment research in France, Luxembourg, Mongolia, Nepal, Senegal, Slovenia, the United States, and the UK.  \n© 2021 Duolingo, Inc  \nContents  \nIntroduction  .......................................................................... 3  \nWhat are the challenges of personalisation for validity?  ................................... 3  \nFairness and standardisation  .......................................................... 4  \nThe opportunities afforded by CAT  ...................................................... 5  \nDoes delightful user design support a more valid test?  .................................... 5  \nWhat does computer-based testing mean for accessibility?  ................................ 7  \nConclusion  ........................................................................... 8  \nReferences  .......................................................................... 9  \n© 2021 Duolingo, Inc  \nIntroduction  \nThe world of educational assessment is experiencing an unprecedented upheaval in both design and delivery. The recent advances in computer-based assessment have provided an unparalleled opportunity torethink the standards, user experience, and vocabulary of assessment validity. Assessments are not simply ways to evaluate knowledge, they are the means to allow test-takers to flourish in their everyday lives. In terms of language assessments, these provide a means to study and reside in the country of the test-takers’choice. With advances in digital assessment, there are new capabilities to interact with test-takers, create virtual scenarios to test their knowledge using engaging items, and remove the administrative barriers afforded by paper-based exams, which have been accentuated during the Covid-19 pandemic. As Mislevy and Haertel (2006) asked,“How can we use these new capabilities to tackle assessment problems we face today?”(p. 6) .  \nIn this paper we describe the new lexicon that digital assessments have introduced into high stakes assessment, such as personalisation, user experience, and accessibility, to explore the opportunities provided by computer-based assessment. Using the Duolingo English Language Test (DET) as an example of innovative assessment design, we argue that the features of high-stakes digital assessments have the potenti","cbCaidKZ2LLpxo2M","https://ap.wps.com/l/cbCaidKZ2LLpxo2M","pdf",312514,3,1,12,"English","en",105,"# Introduction\n# What are the challenges of personalisation for validity?\n# Fairness and standardisation\n# The opportunities afforded by CAT\n# Does delightful user design support a more valid test?\n# What does computer-based testing mean for accessibility?\n# Conclusion\n# References","[{\"question\":\"How do digital-first high-stakes assessments affect test validity standards?\",\"answer\":\"They introduce a new lexicon—such as personalization, user experience, and accessibility—and their features can reduce construct-irrelevant variation. This supports improved test validity through design choices that better target what the test intends to measure.\"},{\"question\":\"What validity challenges arise from personalization, especially compared with standardisation?\",\"answer\":\"Personalisation changes the conditions under which test items are selected and delivered, creating new questions about robustness and fairness. The paper focuses on how departure from standardisation impacts validity.\"},{\"question\":\"How does CAT support more valid assessment design?\",\"answer\":\"Computer-adaptive testing selects items at an appropriate difficulty level for each test-taker. This aims to ensure questions are neither too hard nor too easy, aligning the test process with validity goals.\"},{\"question\":\"Why is accessibility a key consideration in computer-based testing?\",\"answer\":\"The paper explains that computer-based testing changes the accessibility landscape, linking design and delivery choices to inclusive assessment outcomes. It frames accessibility as part of maintaining validity and performance for diverse users.\"}]","Improving Test Validity and Accessibility with Digital-First Assessments | PDF",1787993351,30,{"code":4,"msg":32,"data":33},"ok",{"site_id":25,"language":24,"slug":34,"title":13,"keywords":35,"description":14,"schema_data":36,"social_meta":95,"head_meta":97,"extra_data":99,"updated_unix":29},"improving-test-validity-and-accessibility-with-digital-first-assessments","",{"@graph":37,"@context":94},[38,53,73],{"@type":39,"itemListElement":40},"BreadcrumbList",[41,45,49,51],{"item":42,"name":43,"@type":44,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":46,"name":47,"@type":44,"position":48},"https://docshare.wps.com/document/","Document",2,{"item":50,"name":12,"@type":44,"position":20},"https://docshare.wps.com/document/exam/",{"item":52,"name":13,"@type":44,"position":11},"https://docshare.wps.com/document/improving-test-validity-and-accessibility-with-digital-first-assessments/157994/",{"url":52,"name":13,"@type":54,"image":55,"author":60,"headline":13,"publisher":62,"fileFormat":65,"inLanguage":24,"description":14,"dateModified":66,"datePublished":67,"encodingFormat":65,"isAccessibleForFree":68,"interactionStatistic":69},"DigitalDocument",{"url":56,"@type":57,"width":58,"height":59},"https://docshare.wps.com/thumbnails/improving-test-validity-and-accessibility-with-digital-first-assessments/157994.png","ImageObject",300,407,{"name":9,"@type":61},"Person",{"url":42,"name":63,"@type":64},"DocShare","Organization","application/pdf","2026-09-11","2026-08-29",true,{"@type":70,"interactionType":71,"userInteractionCount":20},"InteractionCounter",{"@type":72},"ViewAction",{"@type":74,"mainEntity":75},"FAQPage",[76,82,86,90],{"name":77,"@type":78,"acceptedAnswer":79},"How do digital-first high-stakes assessments affect test validity standards?","Question",{"text":80,"@type":81},"They introduce a new lexicon—such as personalization, user experience, and accessibility—and their features can reduce construct-irrelevant variation. This supports improved test validity through design choices that better target what the test intends to measure.","Answer",{"name":83,"@type":78,"acceptedAnswer":84},"What validity challenges arise from personalization, especially compared with standardisation?",{"text":85,"@type":81},"Personalisation changes the conditions under which test items are selected and delivered, creating new questions about robustness and fairness. The paper focuses on how departure from standardisation impacts validity.",{"name":87,"@type":78,"acceptedAnswer":88},"How does CAT support more valid assessment design?",{"text":89,"@type":81},"Computer-adaptive testing selects items at an appropriate difficulty level for each test-taker. This aims to ensure questions are neither too hard nor too easy, aligning the test process with validity goals.",{"name":91,"@type":78,"acceptedAnswer":92},"Why is accessibility a key consideration in computer-based testing?",{"text":93,"@type":81},"The paper explains that computer-based testing changes the accessibility landscape, linking design and delivery choices to inclusive assessment outcomes. It frames accessibility as part of maintaining validity and performance for diverse users.","https://schema.org",{"og:url":52,"og:type":96,"og:title":13,"og:site_name":63,"og:description":14},"article",{"robots":98,"canonical":52},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":101},[102,106,110,113,118,123,128,132,137,140,144],{"id":21,"doc_module":4,"doc_module_name":47,"category_name":103,"show_sort_weight":104,"slug":105},"Story & Novel",90,"story-novel",{"id":48,"doc_module":4,"doc_module_name":47,"category_name":107,"show_sort_weight":108,"slug":109},"Literature",80,"literature",{"id":11,"doc_module":4,"doc_module_name":47,"category_name":12,"show_sort_weight":111,"slug":112},70,"exam",{"id":114,"doc_module":4,"doc_module_name":47,"category_name":115,"show_sort_weight":116,"slug":117},5,"Comic",60,"comic",{"id":119,"doc_module":4,"doc_module_name":47,"category_name":120,"show_sort_weight":121,"slug":122},6,"Technology",50,"technology",{"id":124,"doc_module":4,"doc_module_name":47,"category_name":125,"show_sort_weight":126,"slug":127},7,"Healthcare",40,"healthcare",{"id":129,"doc_module":4,"doc_module_name":47,"category_name":130,"show_sort_weight":30,"slug":131},8,"Research & Report","research-report",{"id":133,"doc_module":4,"doc_module_name":47,"category_name":134,"show_sort_weight":135,"slug":136},9,"Religion & Spirituality",20,"religion-spirituality",{"id":135,"doc_module":4,"doc_module_name":47,"category_name":138,"show_sort_weight":135,"slug":139},"World Cup","world-cup",{"id":141,"doc_module":4,"doc_module_name":47,"category_name":142,"show_sort_weight":141,"slug":143},10,"Lifestyle","lifestyle",{"id":145,"doc_module":4,"doc_module_name":47,"category_name":146,"show_sort_weight":114,"slug":147},19,"General","general"]