[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"detail-sidebar-cat-0-en-105":3,"doc-seo-585376-105":59,"doc-detail-585376-en":130},{"code":4,"msg":5,"data":6},0,"success",[7,13,18,23,28,33,38,43,48,51,55],{"id":8,"doc_module":4,"doc_module_name":9,"category_name":10,"show_sort_weight":11,"slug":12},1,"Document","Story & Novel",90,"story-novel",{"id":14,"doc_module":4,"doc_module_name":9,"category_name":15,"show_sort_weight":16,"slug":17},2,"Literature",80,"literature",{"id":19,"doc_module":4,"doc_module_name":9,"category_name":20,"show_sort_weight":21,"slug":22},4,"Exam",70,"exam",{"id":24,"doc_module":4,"doc_module_name":9,"category_name":25,"show_sort_weight":26,"slug":27},5,"Comic",60,"comic",{"id":29,"doc_module":4,"doc_module_name":9,"category_name":30,"show_sort_weight":31,"slug":32},6,"Technology",50,"technology",{"id":34,"doc_module":4,"doc_module_name":9,"category_name":35,"show_sort_weight":36,"slug":37},7,"Healthcare",40,"healthcare",{"id":39,"doc_module":4,"doc_module_name":9,"category_name":40,"show_sort_weight":41,"slug":42},8,"Research & Report",30,"research-report",{"id":44,"doc_module":4,"doc_module_name":9,"category_name":45,"show_sort_weight":46,"slug":47},9,"Religion & Spirituality",20,"religion-spirituality",{"id":46,"doc_module":4,"doc_module_name":9,"category_name":49,"show_sort_weight":46,"slug":50},"World Cup","world-cup",{"id":52,"doc_module":4,"doc_module_name":9,"category_name":53,"show_sort_weight":52,"slug":54},10,"Lifestyle","lifestyle",{"id":56,"doc_module":4,"doc_module_name":9,"category_name":57,"show_sort_weight":24,"slug":58},19,"General","general",{"code":4,"msg":60,"data":61},"ok",{"site_id":62,"language":63,"slug":64,"title":65,"keywords":66,"description":67,"schema_data":68,"social_meta":123,"head_meta":125,"extra_data":127,"updated_unix":129},105,"en","assessing-early-literacy-at-scale-format-effects-in-unsupervised-digital-assessment","Assessing Early Literacy at Scale - Format Effects in Unsupervised Digital Assessment","","Assessing foundational literacy at scale in low-resource settings is challenging because assessor-led testing is costly. Digital platforms provide scalable delivery, yet the impact of item formats under unsupervised conditions for young learners remains insufficiently evidenced. This paper studies a large deployment of EIDU platform assessments in pre-primary classrooms in Kenya, comparing MCQs and VSAQs across 230,000 learners.",{"@graph":69,"@context":122},[70,84,105],{"@type":71,"itemListElement":72},"BreadcrumbList",[73,77,79,82],{"item":74,"name":75,"@type":76,"position":8},"https://docshare.wps.com","Home","ListItem",{"item":78,"name":9,"@type":76,"position":14},"https://docshare.wps.com/document/",{"item":80,"name":40,"@type":76,"position":81},"https://docshare.wps.com/document/research-report/",3,{"item":83,"name":65,"@type":76,"position":19},"https://docshare.wps.com/document/assessing-early-literacy-at-scale-format-effects-in-unsupervised-digital-assessment/585376/",{"url":83,"name":65,"@type":85,"image":86,"author":91,"headline":65,"publisher":94,"fileFormat":97,"inLanguage":63,"description":67,"dateModified":98,"datePublished":99,"encodingFormat":97,"isAccessibleForFree":100,"interactionStatistic":101},"DigitalDocument",{"url":87,"@type":88,"width":89,"height":90},"https://docshare.wps.com/thumbnails/assessing-early-literacy-at-scale-format-effects-in-unsupervised-digital-assessment/585376.png","ImageObject",300,407,{"name":92,"@type":93},"Aria Callaghan","Person",{"url":74,"name":95,"@type":96},"DocShare","Organization","application/pdf","2026-11-10","2026-10-15",true,{"@type":102,"interactionType":103,"userInteractionCount":19},"InteractionCounter",{"@type":104},"ViewAction",{"@type":106,"mainEntity":107},"FAQPage",[108,114,118],{"name":109,"@type":110,"acceptedAnswer":111},"How does item format affect digital assessment outcomes for early literacy in unsupervised settings?","Question",{"text":112,"@type":113},"The study finds that multiple-choice questions (MCQs) produce higher completion and scores than very short answer questions (VSAQs), and item format explains 31% of variance in completed-assessment scores.","Answer",{"name":115,"@type":110,"acceptedAnswer":116},"What does analysis of incorrect VSAQ responses reveal beyond binary scoring?",{"text":117,"@type":113},"Clustering incorrect VSAQ responses shows structured patterns of partial knowledge, confusion about response formats, and emerging phonics understanding that binary correctness scoring obscures.",{"name":119,"@type":110,"acceptedAnswer":120},"Does binary correctness on MCQs and VSAQs align with an independent literacy assessment (IDELA)?",{"text":121,"@type":113},"Binary correctness shows no significant association with IDELA in the linked validation sample (n=384), while categorizing incorrect VSAQ responses by degree of understanding yields a significant directional signal, though generalization beyond the validation set remains open.","https://schema.org",{"og:url":83,"og:type":124,"og:title":65,"og:site_name":95,"og:description":67},"article",{"robots":126,"canonical":83},"index,follow",{"doc_id":128,"site_id":62},585376,1793090352,{"code":4,"msg":5,"data":131},{"doc_id":128,"user_id":132,"nickname":92,"user_avatar":133,"doc_module":4,"category_id":39,"category_name":40,"doc_title":65,"doc_description":67,"doc_content":134,"file_id":135,"file_url":136,"file_type":137,"file_size":138,"view_count":19,"is_deleted":4,"is_public":8,"is_downloadable":8,"audit_status":8,"page_count":39,"language":139,"language_code":63,"site_id":62,"html_lang":63,"table_of_contents":140,"faqs":141,"seo_title":142,"seo_description":67,"update_tm":143,"read_time":46,"preview_page_count":4},962084926284,"https://ap-avatar.wpscdn.com/davatar_29158cc5080c5b710cf443261637dec0","Assessing Early Literacy at Scale: Format Effects in Unsupervised Digital Assessment  \nHarriet Crisp  \nEIDU, Berlin [harriet.crisp@eidu.com](harriet.crisp@eidu.com)  \nAmar Lalwani  \nEIDU, Berlin  \n[amar.lalwani@eidu.com](amar.lalwani@eidu.com)  \nABSTRACT  \nAssessing foundational literacy at scale in low-resource settings is challenging due to the cost of assessor-led testing. Digital platforms offer a scalable alternative, but the performance of different item formats under unsupervised conditions, particularly for young learners, remains underevidenced.  \nThis paper analyses a large-scale deployment of digital assessments delivered through EIDU’s platform-based education programme in pre-primary classrooms in Kenya, comprising multiple-choice questions (MCQs) and very short answer questions (VSAQs) . Across 230,000 learners and 1.8 million item responses, MCQs achieved significantly higher completion and scores than VSAQs, with item format explaining 31% of variance in completed-assessment scores. However, clustering incorrect VSAQ responses revealed highly structured patterns of partial knowledge, responseformat confusion, and emerging phonics understanding that binary scoring entirely obscures.  \nBinary correctness scoring of both MCQs and VSAQs showed no significant association with an independent literacy assessment (IDELA; linked sample n = 384), but classifying incorrect VSAQ responses by degree of understanding uncovered a significant directional signal, though whether this generalises beyond the small validation sample remains an open question. The results highlight a trade-off between MCQ engagement and the richer learner signal in VSAQ errors, motivating further validation of their diagnostic value.  \nKeywords  \nDigital assessment, Foundational literacy, Very short answer questions (VSAQs), Unsupervised learning environments, Low-resource settings  \n1. INTRODUCTION & MOTIVATION  \nIt is widely recognised that foundational literacy and numeracy skills are essential predictors of future academic  \nHarriet Crisp, and Amar Lalwani. Assessing Early Literacy at Scale: Format Effects in Unsupervised Digital Assessment. In Anthony Botelho, Maria Mercedes T. Rodrigo, Adish Singla, Hiroaki Ogata, Hyojeong So, and Young Hoan Cho (eds.) Proceedings of the 19th International Conference on Educational Data Mining, Seoul, Republic of Korea, June, 2026, pp. 901–908. International Educational Data Mining Society (2026) .  \n© 2026 Copyright is held by the author(s) . This work is distributed under the Creative Commons Attribution NonCommercial NoDerivatives 4.0 International (CC BY-NC-ND 4.0) license. [https://doi.org/10.5281/zenodo.21040036](https://doi.org/10.5281/zenodo.21040036)  \nsuccess [6,22], therefore accurate assessment of these skills is essential. This is of particular importance to learners in low- and middle-income countries (LMICs), who consistently demonstrate substantial gaps in these critical areas. National assessments in a Sub-Saharan African country indicate that 60% of Grade 3 learners fail to achieve minimum proficiency levels in reading [29], compared to less than 10% in high income countries [13] .  \nMany widely used child assessment instruments were developed and normed in high-income settings, raising concerns about contextual fit, feasibility, and interpretability when transferred to low-resource environments [7, 8, 14] . In response, LMIC-focused tools have emerged for early learning measurement, including population- and program-evaluation instruments such as IDELA [18] and MELQO [21], and foundational skill assessments such as the Early Grade Reading Assessment (EGRA) and Early Grade Mathematics Assessment (EGMA) [5,23] . Both EGRA and EGMA have been widely adopted in low-and middle-income contexts for system diagnosis and program evaluation.  \nHowever, because these assessments are typically assessorled, one-to-one, and time-intensive, they are logistically infeasible at scale, constraining their use fo","cbCaikSA2j8UfbAX","https://ap.wps.com/l/cbCaikSA2j8UfbAX","pdf",2329011,"English","# Abstract\n# 1. Introduction & Motivation\n# 2. Related Work and Background\n# 3. Study Design and Data\n# 4. Results: Completion, Scores, and Error Patterns\n# 5. Validation Against IDELA\n# 6. Discussion and Implications\n# 7. Conclusion","[{\"question\":\"How does item format affect digital assessment outcomes for early literacy in unsupervised settings?\",\"answer\":\"The study finds that multiple-choice questions (MCQs) produce higher completion and scores than very short answer questions (VSAQs), and item format explains 31% of variance in completed-assessment scores.\"},{\"question\":\"What does analysis of incorrect VSAQ responses reveal beyond binary scoring?\",\"answer\":\"Clustering incorrect VSAQ responses shows structured patterns of partial knowledge, confusion about response formats, and emerging phonics understanding that binary correctness scoring obscures.\"},{\"question\":\"Does binary correctness on MCQs and VSAQs align with an independent literacy assessment (IDELA)?\",\"answer\":\"Binary correctness shows no significant association with IDELA in the linked validation sample (n=384), while categorizing incorrect VSAQ responses by degree of understanding yields a significant directional signal, though generalization beyond the validation set remains open.\"}]","Assessing Early Literacy at Scale - Format Effects in Unsupervised Digital Assessment | PDF",1792054552]