[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-85646-en":3,"doc-seo-85646-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},85646,4398048949847,"Eliana","https://ap-avatar.wpscdn.com/avatar/400002536579ef2da7f?_k=1778318612642679267",8,"Research & Report","Cognitive Episodes in LLM Reasoning Traces Enable Interpretable Human Item Difficulty Prediction","Human item difficulty prediction is a core requirement for fair and effective educational assessment, yet existing methods often rely on costly calibration or item text features and provide limited interpretability into the cognition behind difficulty. This work models difficulty as a consequence of human problem-solving burden and leverages reasoning-oriented LLM traces. It introduces cognitive episodes that map trace segments to functional states and proposes Epi2Diff to extract compact process features, combining them with semantic item representations for accurate, interpretable prediction.","arXiv :2606 .28 186v2 [ cs .CL] 11 Jul 2026  \nCognitive Episodes in LLM Reasoning Traces Enable Interpretable Human Item Difficulty Prediction  \nChenguang Wang∗ , 1 , Ming Li∗ ,2 ,3 , Xinyue Zeng 1 , Zhuochun Li4 , Hong Jiao2 , Tianyi Zhou3 , Dawei Zhou 1 1Virginia Tech 2 University of Maryland 3 MBZUAI 4 University of Pittsburgh  \n∗ Co-first Author  \nPredicting human item difficulty is a central challenge in educational assessment, where accurate estimates are critical for fairness and effective test construction. Existing approaches typically rely on costly human calibration or item-level textual representations, offering limited interpretable evidence about the cognitive processes that make an item difficult. We argue that item difficulty should be viewed not only as a property of item text, but also as an observable consequence of the problemsolving burden induced by the item. Yet, this burden from human is difficult to observe directly or analyze at scale. Reasoning-oriented large language models (LLMs) offer scalable process evidence through explicit reasoning traces, but raw traces are often lengthy and unstructured. To organize this evidence, we introduce cognitive episodes, grounded in Schoenfeld’s account of problem solving, which group trace segments into functional problem-solving states. Building on this representation, we propose Epi2Diff (Episode to Difficulty), a framework that converts model-generated reasoning traces into episode sequences and extracts compact features capturing the scale, allocation, and transitions of reasoning effort. These process features are combined with semantic item representations for human item difficulty prediction. Experiments on four real-world datasets show that Epi2Diff consistently outperforms strong baselines, including fine-tuned SLMs, LLM in-context learning, and supervised LLM adaptation. On the SAT-derived classification benchmarks, Epi2Diff achievesan 8 . 1% average relative gain over supervised LLM fine-tuning baselines. Further analyses show that harder items induce more effortful, iterative, and implementation-centered episode dynamics rather than merely longer responses. These results establish cognitive episodes as a predictive and interpretable process representation for human item difficulty. The anonymized code is available at [https://anonymous.4open.science/r/Epi2Diff-B10A](https://anonymous.4open.science/r/Epi2Diff-B10A).  \nWebsite: [https://github.com/c-steve-wang/Epi2Diff](https://github.com/c-steve-wang/Epi2Diff)  \nAuthor E-mails: {cswang, [dzhou}@vt.edu](dzhou}@vt.edu), [minglii@umd.edu](minglii@umd.edu), [tianyi.zhou@mbzuai.ac.ae](tianyi.zhou@mbzuai.ac.ae)  \n1 Introduction  \nLarge-scale assessment is a core evaluation mechanism in domains such as college admissions, language testing, and medical licensure (Hambleton et al. , 1991 ; Eignor, 2013 ; Hsu et al. , 2018 ; AlKhuzaey et al. , 2021) . In these settings, item difficulty prediction is important for constructing assessments with well-calibrated difficulty distributions and precise measurement along the ability scale (AlKhuzaey et al. , 2021 ; Parry, 2020) . Traditionally, item difficulty has been estimated from examinee responses using classical psychometric frameworks such as Classical Test Theory (CTT) and Item Response Theory (IRT) from responses collected during field testing (Hsu et al. , 2018 ; DeMars, 2010 ; AlKhuzaey et al. , 2024 ; Benedetto, 2023) . Although effective, these approaches require substantial pretesting and are therefore costly, time-consuming, and difficult to scale.  \nIn response to these limitations, prior work has explored automated approaches for predicting item difficulty. Existing approaches mainly include feature-based methods and end-to-end fine-tuning methods (Peters et al. , 2025 ; AlKhuzaey et al. , 2024 ; Benedetto, 2023) . Feature-based methods are generally more interpretable but depend on representation quality, whereas end-to-end fine-tuning methods, such as f","cbCaifiiF9802hRJ","https://ap.wps.com/l/cbCaifiiF9802hRJ","pdf",2624472,1,31,"English","en",105,"# Introduction\n## Human item difficulty prediction challenges\n## Limitations of existing approaches\n## LLM-based directions and gaps","[{\"question\":\"Why is predicting human item difficulty important in educational assessment?\",\"answer\":\"Accurate estimates support well-calibrated difficulty distributions and precise measurement along the ability scale, which is essential for fairness and effective test construction.\"},{\"question\":\"What limitation do prior item-difficulty methods have regarding interpretability?\",\"answer\":\"Many methods depend on expensive human calibration or item-level text representations, offering limited interpretable evidence about the cognitive processes that make items difficult.\"},{\"question\":\"How does Epi2Diff improve human item difficulty prediction using LLM reasoning?\",\"answer\":\"Epi2Diff converts model-generated reasoning traces into cognitive-episode sequences, extracts compact process features (scale, allocation, transitions of reasoning effort), and combines them with semantic item representations to make more accurate predictions.\"}]",1784205331,78,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"cognitive-episodes-in-llm-reasoning-traces-enable-interpretable-human-item-difficulty-prediction","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/cognitive-episodes-in-llm-reasoning-traces-enable-interpretable-human-item-difficulty-prediction/85646/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why is predicting human item difficulty important in educational assessment?","Question",{"text":75,"@type":76},"Accurate estimates support well-calibrated difficulty distributions and precise measurement along the ability scale, which is essential for fairness and effective test construction.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What limitation do prior item-difficulty methods have regarding interpretability?",{"text":80,"@type":76},"Many methods depend on expensive human calibration or item-level text representations, offering limited interpretable evidence about the cognitive processes that make items difficult.",{"name":82,"@type":73,"acceptedAnswer":83},"How does Epi2Diff improve human item difficulty prediction using LLM reasoning?",{"text":84,"@type":76},"Epi2Diff converts model-generated reasoning traces into cognitive-episode sequences, extracts compact process features (scale, allocation, transitions of reasoning effort), and combines them with semantic item representations to make more accurate predictions.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]