[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"detail-sidebar-cat-0-en-105":3,"doc-seo-577871-105":59,"doc-detail-577871-en":131},{"code":4,"msg":5,"data":6},0,"success",[7,13,18,23,28,33,38,43,48,51,55],{"id":8,"doc_module":4,"doc_module_name":9,"category_name":10,"show_sort_weight":11,"slug":12},1,"Document","Story & Novel",90,"story-novel",{"id":14,"doc_module":4,"doc_module_name":9,"category_name":15,"show_sort_weight":16,"slug":17},2,"Literature",80,"literature",{"id":19,"doc_module":4,"doc_module_name":9,"category_name":20,"show_sort_weight":21,"slug":22},4,"Exam",70,"exam",{"id":24,"doc_module":4,"doc_module_name":9,"category_name":25,"show_sort_weight":26,"slug":27},5,"Comic",60,"comic",{"id":29,"doc_module":4,"doc_module_name":9,"category_name":30,"show_sort_weight":31,"slug":32},6,"Technology",50,"technology",{"id":34,"doc_module":4,"doc_module_name":9,"category_name":35,"show_sort_weight":36,"slug":37},7,"Healthcare",40,"healthcare",{"id":39,"doc_module":4,"doc_module_name":9,"category_name":40,"show_sort_weight":41,"slug":42},8,"Research & Report",30,"research-report",{"id":44,"doc_module":4,"doc_module_name":9,"category_name":45,"show_sort_weight":46,"slug":47},9,"Religion & Spirituality",20,"religion-spirituality",{"id":46,"doc_module":4,"doc_module_name":9,"category_name":49,"show_sort_weight":46,"slug":50},"World Cup","world-cup",{"id":52,"doc_module":4,"doc_module_name":9,"category_name":53,"show_sort_weight":52,"slug":54},10,"Lifestyle","lifestyle",{"id":56,"doc_module":4,"doc_module_name":9,"category_name":57,"show_sort_weight":24,"slug":58},19,"General","general",{"code":4,"msg":60,"data":61},"ok",{"site_id":62,"language":63,"slug":64,"title":65,"keywords":66,"description":67,"schema_data":68,"social_meta":124,"head_meta":126,"extra_data":128,"updated_unix":130},105,"en","disengagement-matters-a-response-time-informed-approach-to-scoring-lowstakes-assessments","Disengagement matters - A response-time informed approach to scoring lowstakes assessments","","Low-stakes assessments in K-12 monitoring support are threatened by disengaged test-taking behaviors, especially rapid guessing and idling, which introduce construct-irrelevant variance and reduce score validity and reliability. The study applies a Nested Logit Model to combine response accuracy with response times, using data from a U.S. low-stakes reading assessment (n=27,556; grades 5-8). Six scoring approaches are compared, including dichotomous, effort-moderated, and RT-informed nominal scoring. Results show nominal scoring separating rapid guesses from idles improves fit, precision, and behavioral interpretability, and latent classes reveal three engagement profiles.",{"@graph":69,"@context":123},[70,84,106],{"@type":71,"itemListElement":72},"BreadcrumbList",[73,77,79,82],{"item":74,"name":75,"@type":76,"position":8},"https://docshare.wps.com","Home","ListItem",{"item":78,"name":9,"@type":76,"position":14},"https://docshare.wps.com/document/",{"item":80,"name":40,"@type":76,"position":81},"https://docshare.wps.com/document/research-report/",3,{"item":83,"name":65,"@type":76,"position":19},"https://docshare.wps.com/document/disengagement-matters-a-response-time-informed-approach-to-scoring-lowstakes-assessments/577871/",{"url":83,"name":65,"@type":85,"image":86,"author":91,"headline":65,"publisher":94,"fileFormat":97,"inLanguage":63,"description":67,"dateModified":98,"datePublished":99,"encodingFormat":97,"isAccessibleForFree":100,"interactionStatistic":101},"DigitalDocument",{"url":87,"@type":88,"width":89,"height":90},"https://docshare.wps.com/thumbnails/disengagement-matters-a-response-time-informed-approach-to-scoring-lowstakes-assessments/577871.png","ImageObject",300,407,{"name":92,"@type":93},"Emma Wilson","Person",{"url":74,"name":95,"@type":96},"DocShare","Organization","application/pdf","2026-11-16","2026-10-15",true,{"@type":102,"interactionType":103,"userInteractionCount":105},"InteractionCounter",{"@type":104},"ViewAction",12,{"@type":107,"mainEntity":108},"FAQPage",[109,115,119],{"name":110,"@type":111,"acceptedAnswer":112},"Why do disengaged behaviors harm low-stakes assessment validity?","Question",{"text":113,"@type":114},"Disengaged behaviors such as rapid guessing and idling lower performance and add construct-irrelevant variance, which can bias item statistics and distort validity and reliability.","Answer",{"name":116,"@type":111,"acceptedAnswer":117},"How does the study use response time (RT) in scoring?",{"text":118,"@type":114},"RTs are integrated with item responses via a Nested Logit Model, allowing scoring approaches to reflect engagement indicated by unusually fast (guessing) or slow (idling) response patterns.",{"name":120,"@type":111,"acceptedAnswer":121},"Which scoring approach performed best in the study?",{"text":122,"@type":114},"Nominal scoring approaches that distinguish rapid guesses from idle responses produced superior model fit and increased measurement precision compared with dichotomous scoring methods.","https://schema.org",{"og:url":83,"og:type":125,"og:title":65,"og:site_name":95,"og:description":67},"article",{"robots":127,"canonical":83},"index,follow",{"doc_id":129,"site_id":62},577871,1792170633,{"code":4,"msg":5,"data":132},{"doc_id":129,"user_id":133,"nickname":92,"user_avatar":134,"doc_module":4,"category_id":39,"category_name":40,"doc_title":65,"doc_description":67,"doc_content":135,"file_id":136,"file_url":137,"file_type":138,"file_size":139,"view_count":105,"is_deleted":4,"is_public":8,"is_downloadable":8,"audit_status":8,"page_count":140,"language":141,"language_code":63,"site_id":62,"html_lang":63,"table_of_contents":142,"faqs":143,"seo_title":144,"seo_description":67,"update_tm":145,"read_time":146,"preview_page_count":4},3848291630094,"https://eur-avatar.wpscdn.com/davatar_085a072bc5b1113ac321206ff7593b45","Disengagement matters: A response-time informed approach to scoring lowstakes assessments  \nOkan Bulut1*, Joyce Xinle Liu2, Hatice Cigdem Bulut3  \n1University of Alberta, Faculty of Education, Centre for Research in Applied Measurement and Evaluation 2University of Alberta, Faculty of Education, Measurement, Evaluation, and Data Science  \n3Northern Alberta Institute of Technology, Education Insights, Data and Research  \nARTICLE HISTORY  \nReceived: June 16, 2025  \nAccepted: Nov. 13, 2025  \nKEYWORDS  \nLow-stakes testing, Disengagement, Rapid guessing, Idling,  \nItem response modeling.  \nAbstract: Low-stakes assessments in K-12 education play a crucial role in monitoring student progress, yet their validity is often compromised by disengaged test-taking behaviors, such as rapid guessing and idling. This study introduces a novel application of the Nested Logit Model (NLM) to integrate test-taking engagement, as indicated by response times (RTs), with item responses to improve ability estimation. Using data from a low-stakes reading assessment in the United States (n = 27,556 students in grades 5 to 8), we compared six scoring approaches, including traditional dichotomous scoring, effort-moderated scoring, and nominal scoring that categorized responses based on RT-informed engagement. Our results demonstrated that nominal scoring approaches, particularly those distinguishing rapid guesses and idle responses, yielded superior model fit, increased measurement precision, and provided nuanced insights into examinee behaviors compared to dichotomous scoring methods. Latent class analysis further identified three distinct engagement profiles—effortful responders, rapid guessers, and idle responders—highlighting the need to address both rapid and idle behaviors in modeling. This study emphasizes the value of leveraging RT data to enhance the accuracy of low-stakes assessments while preserving response information. Findings also suggest the NLM framework as a practical and accessible tool for researchers and practitioners seeking to address disengaged behaviors and ensure the reliability of low-stakes assessments.  \nINTRODUCTION  \nIn K-12 settings, low-stakes assessments are often used to monitor students’ learning progress throughout a school year. Teachers use the scores from these assessments to identify areas where students need more support, and education administrators may use them for broader analysis or accountability purposes (Filiasov & Sweetman, 2023; Finn, 2015) . For students, low-stakes assessment may not be considered important because there are no personal benefits or consequences tied to their performance (Wise, 2006) . As a result, some students may not invest sufficient effort in responding to items, which could compromise the validity and reliability of test scores (Eklöf, 2006; Wise, 2015) . Non-effortful or disengaged test-taking behavior is a widely recognized concern in the measurement literature, particularly in low-  \n*Corresponding author at: Okan BULUT 􀀍 [bulut@ualberta.ca](bulut@ualberta.ca) 􀁽 Centre for Research in Applied Measurement and Evaluation, University of Alberta, 6-110 Education Centre North, 11210 87 Ave NW, Edmonton, AB T6G 2G5 Canada  \nThe copyright of the published article belongs to its author under CC BY 4.0 license. To view a copy of this licence, visit [http://creativecommons.org/licenses/by/4.0/](http://creativecommons.org/licenses/by/4.0/)  \n[e-ISSN: 2148-7456](e-ISSN: 2148-7456)  \nstakes testing contexts where low motivation is more prevalent (Finn, 2015) . Disengaged behavior lowers performance and introduces construct-irrelevant variance to test scores, which could bias item statistics and distort validity and reliability measures (Finn, 2015; Rios & Deng, 2022; Wise & DeMars, 2009) . Thus, there have been increasing calls for analytical methods that account for disengaged test-taking to ensure score validity (Finn, 2015) .  \nWith the advent of computerized testing and the ability to rec","cbCaiiA8r4WYI6PU","https://ap.wps.com/l/cbCaiiA8r4WYI6PU","pdf",699022,22,"English","# Introduction\n## Disengagement in low-stakes testing\n## Response time as an engagement signal\n## Prior RT-based approaches\n# Method and scoring comparison\n## Nested Logit Model with engagement\n## Scoring approaches and evaluation metrics\n## Latent engagement profiles","[{\"question\":\"Why do disengaged behaviors harm low-stakes assessment validity?\",\"answer\":\"Disengaged behaviors such as rapid guessing and idling lower performance and add construct-irrelevant variance, which can bias item statistics and distort validity and reliability.\"},{\"question\":\"How does the study use response time (RT) in scoring?\",\"answer\":\"RTs are integrated with item responses via a Nested Logit Model, allowing scoring approaches to reflect engagement indicated by unusually fast (guessing) or slow (idling) response patterns.\"},{\"question\":\"Which scoring approach performed best in the study?\",\"answer\":\"Nominal scoring approaches that distinguish rapid guesses from idle responses produced superior model fit and increased measurement precision compared with dichotomous scoring methods.\"}]","Disengagement matters - A response-time informed approach to scoring lowstakes assessments | PDF",1792032572,55]