[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-seo-578932-105":3,"detail-sidebar-cat-0-en-105":81,"doc-detail-578932-en":130},{"code":4,"msg":5,"data":6},0,"ok",{"site_id":7,"language":8,"slug":9,"title":10,"keywords":11,"description":12,"schema_data":13,"social_meta":74,"head_meta":76,"extra_data":78,"updated_unix":80},105,"en","data-plus-theory-equals-codebook-leveraging-llms-for-human-ai-codebook-development","Data Plus Theory Equals Codebook - Leveraging LLMs for Human-AI Codebook Development","","Explores how GPT-4o can support theory-informed qualitative codebook development across two learning-science contexts. The first study uses a three-step workflow grounded in Self-Regulated Learning theories, think-aloud data, and human refinement to assess codebook quality and alignment. The second study applies Interest Development’s four phases in a STEM game setting and compares four prompting strategies, finding that naming theories without full references maximizes practicality, while full papers improve theoretical fit but reduce applicability. Results position foundation-model prompting as a useful partner for qualitative research when carefully designed.",{"@graph":14,"@context":73},[15,34,56],{"@type":16,"itemListElement":17},"BreadcrumbList",[18,23,27,31],{"item":19,"name":20,"@type":21,"position":22},"https://docshare.wps.com","Home","ListItem",1,{"item":24,"name":25,"@type":21,"position":26},"https://docshare.wps.com/document/","Document",2,{"item":28,"name":29,"@type":21,"position":30},"https://docshare.wps.com/document/research-report/","Research & Report",3,{"item":32,"name":10,"@type":21,"position":33},"https://docshare.wps.com/document/data-plus-theory-equals-codebook-leveraging-llms-for-human-ai-codebook-development/578932/",4,{"url":32,"name":10,"@type":35,"image":36,"author":41,"headline":10,"publisher":44,"fileFormat":47,"inLanguage":8,"description":12,"dateModified":48,"datePublished":49,"encodingFormat":47,"isAccessibleForFree":50,"interactionStatistic":51},"DigitalDocument",{"url":37,"@type":38,"width":39,"height":40},"https://docshare.wps.com/thumbnails/data-plus-theory-equals-codebook-leveraging-llms-for-human-ai-codebook-development/578932.png","ImageObject",300,407,{"name":42,"@type":43},"Noah","Person",{"url":19,"name":45,"@type":46},"DocShare","Organization","application/pdf","2026-11-16","2026-10-15",true,{"@type":52,"interactionType":53,"userInteractionCount":55},"InteractionCounter",{"@type":54},"ViewAction",8,{"@type":57,"mainEntity":58},"FAQPage",[59,65,69],{"name":60,"@type":61,"acceptedAnswer":62},"What problem does the paper address in qualitative codebook development?","Question",{"text":63,"@type":64},"It addresses the gap where existing LLM-based approaches may overlook the theoretical grounding needed for qualitative analysis, especially when manual review of large datasets is impractical.","Answer",{"name":66,"@type":61,"acceptedAnswer":67},"How does the first study evaluate GPT-4o for codebook generation?",{"text":68,"@type":64},"It uses a three-step approach based on Self-Regulated Learning theories, think-aloud data, and human refinement to assess whether GPT-4o can generate high-quality, theory-aligned codebooks.",{"name":70,"@type":61,"acceptedAnswer":71},"Which prompting strategy performed best in the second study and why?",{"text":72,"@type":64},"Naming the theory without including full references produced the most practical and usable codebook, while supplying full theory papers improved alignment but reduced applicability.","https://schema.org",{"og:url":32,"og:type":75,"og:title":10,"og:site_name":45,"og:description":12},"article",{"robots":77,"canonical":32},"index,follow",{"doc_id":79,"site_id":7},578932,1792210817,{"code":4,"msg":82,"data":83},"success",[84,88,92,96,101,106,111,114,119,122,126],{"id":22,"doc_module":4,"doc_module_name":25,"category_name":85,"show_sort_weight":86,"slug":87},"Story & Novel",90,"story-novel",{"id":26,"doc_module":4,"doc_module_name":25,"category_name":89,"show_sort_weight":90,"slug":91},"Literature",80,"literature",{"id":33,"doc_module":4,"doc_module_name":25,"category_name":93,"show_sort_weight":94,"slug":95},"Exam",70,"exam",{"id":97,"doc_module":4,"doc_module_name":25,"category_name":98,"show_sort_weight":99,"slug":100},5,"Comic",60,"comic",{"id":102,"doc_module":4,"doc_module_name":25,"category_name":103,"show_sort_weight":104,"slug":105},6,"Technology",50,"technology",{"id":107,"doc_module":4,"doc_module_name":25,"category_name":108,"show_sort_weight":109,"slug":110},7,"Healthcare",40,"healthcare",{"id":55,"doc_module":4,"doc_module_name":25,"category_name":29,"show_sort_weight":112,"slug":113},30,"research-report",{"id":115,"doc_module":4,"doc_module_name":25,"category_name":116,"show_sort_weight":117,"slug":118},9,"Religion & Spirituality",20,"religion-spirituality",{"id":117,"doc_module":4,"doc_module_name":25,"category_name":120,"show_sort_weight":117,"slug":121},"World Cup","world-cup",{"id":123,"doc_module":4,"doc_module_name":25,"category_name":124,"show_sort_weight":123,"slug":125},10,"Lifestyle","lifestyle",{"id":127,"doc_module":4,"doc_module_name":25,"category_name":128,"show_sort_weight":97,"slug":129},19,"General","general",{"code":4,"msg":82,"data":131},{"doc_id":79,"user_id":132,"nickname":42,"user_avatar":133,"doc_module":4,"category_id":55,"category_name":29,"doc_title":10,"doc_description":12,"doc_content":134,"file_id":135,"file_url":136,"file_type":137,"file_size":138,"view_count":55,"is_deleted":4,"is_public":22,"is_downloadable":22,"audit_status":22,"page_count":139,"language":140,"language_code":8,"site_id":7,"html_lang":8,"table_of_contents":141,"faqs":142,"seo_title":143,"seo_description":12,"update_tm":144,"read_time":145,"preview_page_count":4},8796095462418,"https://ap-avatar.wpscdn.com/avatar/80000253c1241d02b47?x-image-process=image/resize,m_fixed,w_180,h_180&k=1778826106357471780","Data Plus Theory Equals Codebook: Leveraging LLMs for Human-AI Codebook Development  \nAndres Felipe Zambrano University of Pennsylvania Philadelphia, PA, USA [azamb13@upenn.edu](azamb13@upenn.edu)  \nZhanlan Wei  \nUniversity of Pennsylvania Philadelphia, PA, USA [zhanlanw@upenn.edu](zhanlanw@upenn.edu)  \nJiayi Zhang  \nUniversity of Pennsylvania Philadelphia, PA, USA [joycez@upenn.edu](joycez@upenn.edu)  \nRyan S. Baker  \nAdelaide University Adelaide , SA, Australia [ryanshaunbaker@gmail.com](ryanshaunbaker@gmail.com)  \nJaclyn Ocumpaugh University of Houston Houston , TX, USA [jlocumpaugh@gmail.com](jlocumpaugh@gmail.com)  \nAmanda Barany University of Pennsylvania Philadelphia, PA, USA [amanda.barany@gmail.com](amanda.barany@gmail.com)  \nXiner Liu  \nUniversity of Pennsylvania Philadelphia, PA, USA [xiner@upenn.edu](xiner@upenn.edu)  \nJeffrey Ginger  \nUniversity of Illinois at Urbana-Champaign Champaign, IL, USA  \n[ginger@illinois.edu](ginger@illinois.edu)  \nLuc Paquette  \nUniversity of Illinois at Urbana-Champaign Champaign, IL, USA [luc.paquette@gmail.com](luc.paquette@gmail.com)  \nYiqiu Zhou  \nUniversity of Illinois at Urbana-Champaign Champaign, IL, USA  \n[yiqiuz3@illinois.edu](yiqiuz3@illinois.edu)  \nConrad Borchers Carnegie Mellon University Pittsburgh, PA, USA[cborcher@cs.cmu.edu](cborcher@cs.cmu.edu)  \nRecent research has explored the use of Large Language Models (LLMs) to develop qualitative codebooks, mainly for inductive work with large datasets, where manual review is impractical. Although these efforts  \n25 Journal of Educational Data Mining, Volume 18, No 1, 2026  \nshow promise, they often neglect the theoretical grounding essential to many types of qualitative analysis. This paper investigates the potential of GPT-4o to support theory-informed codebook development across two educational contexts. In the first study, we employ a three-step approach—drawing on Winne & Hadwin’sand Zimmerman’s Self-Regulated Learning (SRL) theories, think-aloud data, and human refinement—to evaluate GPT-4o’s ability to generate high-quality, theory-aligned codebooks. Results indicate that GPT-4o can effectively leverage its knowledge base to identify SRL constructs reflected in student problem-solving behavior. In the second study, we extend this approach to a STEM game-based learning context guided by Hidi & Renninger’s four-phase model of Interest Development. We compare four prompting strategies: no theories provided, theories named, full references given, and full-text theory papers supplied. Human evaluations show that naming the theory without including full references produced the most practical and usable codebook, while supplying full papers to the prompt enhanced theoretical alignment but reduced applicability. These findings suggest that GPT-4o can be a valuable partner in theory-driven qualitative research when grounded in well-established frameworks, but that attention to prompt design is required. Our results show that widely available foundation models—trained on large-scale open web and licensed datasets—can effectively distill established educational theories to support qualitative research and codebook development. The code for our codebook development process and all the employed prompts and codebooks produced by GPTare available for replication purposes at: [https://osf.io/g3z4x](https://osf.io/g3z4x)  \nKeywords: large language models, qualitative codebooks, interest development, self-regulated learning, thematic analysis, codebook development  \n1. INTRODUCTION  \nQualitative analysis has been used in the learning sciences to systematically examine large volumes of rich, multimodal data (e.g., interview transcripts, field notes, and video recordings) to identify patterns and elaborate insightful and trustworthy interpretations (Gibbs, 2018) . While earlier work in Educational Data Mining (EDM) has leveraged qualitative research as a mixedmethods supplement to quantitative results, more recent work has sought to quanti","cbCaisF3K4BlovZC","https://ap.wps.com/l/cbCaisF3K4BlovZC","pdf",711979,41,"English","# INTRODUCTION\n## Qualitative analysis and coding\n## Inductive vs. deductive codebook development\n## Need for theory-informed, scalable approaches","[{\"question\":\"What problem does the paper address in qualitative codebook development?\",\"answer\":\"It addresses the gap where existing LLM-based approaches may overlook the theoretical grounding needed for qualitative analysis, especially when manual review of large datasets is impractical.\"},{\"question\":\"How does the first study evaluate GPT-4o for codebook generation?\",\"answer\":\"It uses a three-step approach based on Self-Regulated Learning theories, think-aloud data, and human refinement to assess whether GPT-4o can generate high-quality, theory-aligned codebooks.\"},{\"question\":\"Which prompting strategy performed best in the second study and why?\",\"answer\":\"Naming the theory without including full references produced the most practical and usable codebook, while supplying full theory papers improved alignment but reduced applicability.\"}]","Data Plus Theory Equals Codebook - Leveraging LLMs for Human-AI Codebook Development | PDF",1792034672,103]