[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-seo-579032-105":3,"detail-sidebar-cat-0-en-105":81,"doc-detail-579032-en":130},{"code":4,"msg":5,"data":6},0,"ok",{"site_id":7,"language":8,"slug":9,"title":10,"keywords":11,"description":12,"schema_data":13,"social_meta":74,"head_meta":76,"extra_data":78,"updated_unix":80},105,"en","integrating-topic-modeling-and-llm-prompt-engineering-into-a-human-driven-approach-to-analyze-interview-transcripts","Integrating Topic Modeling and LLM Prompt Engineering into a Human-driven Approach to Analyze Interview Transcripts","","Topic modeling is a widely used unsupervised method for extracting latent themes from large text corpora, yet interpretability often depends on human judgment, limiting transparency and reproducibility. Large language models and prompt engineering may improve interpretability and scalability of topic modeling results. The study proposes a hybrid human-in-the-loop framework combining topic modeling, LLM prompting, and human-derived codes for rigorous qualitative analysis of focus group interviews with 13 U.S. teachers. Findings show improved construct alignment, thematic stability, and methodological transparency while preserving qualitative context richness through necessary human oversight.",{"@graph":14,"@context":73},[15,34,56],{"@type":16,"itemListElement":17},"BreadcrumbList",[18,23,27,31],{"item":19,"name":20,"@type":21,"position":22},"https://docshare.wps.com","Home","ListItem",1,{"item":24,"name":25,"@type":21,"position":26},"https://docshare.wps.com/document/","Document",2,{"item":28,"name":29,"@type":21,"position":30},"https://docshare.wps.com/document/research-report/","Research & Report",3,{"item":32,"name":10,"@type":21,"position":33},"https://docshare.wps.com/document/integrating-topic-modeling-and-llm-prompt-engineering-into-a-human-driven-approach-to-analyze-interview-transcripts/579032/",4,{"url":32,"name":10,"@type":35,"image":36,"author":41,"headline":10,"publisher":44,"fileFormat":47,"inLanguage":8,"description":12,"dateModified":48,"datePublished":49,"encodingFormat":47,"isAccessibleForFree":50,"interactionStatistic":51},"DigitalDocument",{"url":37,"@type":38,"width":39,"height":40},"https://docshare.wps.com/thumbnails/integrating-topic-modeling-and-llm-prompt-engineering-into-a-human-driven-approach-to-analyze-interview-transcripts/579032.png","ImageObject",300,407,{"name":42,"@type":43},"Noah","Person",{"url":19,"name":45,"@type":46},"DocShare","Organization","application/pdf","2026-11-15","2026-10-15",true,{"@type":52,"interactionType":53,"userInteractionCount":55},"InteractionCounter",{"@type":54},"ViewAction",10,{"@type":57,"mainEntity":58},"FAQPage",[59,65,69],{"name":60,"@type":61,"acceptedAnswer":62},"What problem does the study address in topic modeling for qualitative research?","Question",{"text":63,"@type":64},"It addresses limited transparency and reproducibility when topic interpretability depends heavily on human judgment.","Answer",{"name":66,"@type":61,"acceptedAnswer":67},"How does the proposed framework combine topic modeling, LLM prompting, and human input?",{"text":68,"@type":64},"It uses semantic clustering, LLM-assisted topic labeling, and iterative codebook refinement supported by human-derived codes.",{"name":70,"@type":61,"acceptedAnswer":71},"What benefits do the findings report for using this human-in-the-loop approach?",{"text":72,"@type":64},"The approach supports construct alignment, thematic stability, and methodological transparency while preserving the contextual richness of qualitative data.","https://schema.org",{"og:url":32,"og:type":75,"og:title":10,"og:site_name":45,"og:description":12},"article",{"robots":77,"canonical":32},"index,follow",{"doc_id":79,"site_id":7},579032,1792189894,{"code":4,"msg":82,"data":83},"success",[84,88,92,96,101,106,111,115,120,123,126],{"id":22,"doc_module":4,"doc_module_name":25,"category_name":85,"show_sort_weight":86,"slug":87},"Story & Novel",90,"story-novel",{"id":26,"doc_module":4,"doc_module_name":25,"category_name":89,"show_sort_weight":90,"slug":91},"Literature",80,"literature",{"id":33,"doc_module":4,"doc_module_name":25,"category_name":93,"show_sort_weight":94,"slug":95},"Exam",70,"exam",{"id":97,"doc_module":4,"doc_module_name":25,"category_name":98,"show_sort_weight":99,"slug":100},5,"Comic",60,"comic",{"id":102,"doc_module":4,"doc_module_name":25,"category_name":103,"show_sort_weight":104,"slug":105},6,"Technology",50,"technology",{"id":107,"doc_module":4,"doc_module_name":25,"category_name":108,"show_sort_weight":109,"slug":110},7,"Healthcare",40,"healthcare",{"id":112,"doc_module":4,"doc_module_name":25,"category_name":29,"show_sort_weight":113,"slug":114},8,30,"research-report",{"id":116,"doc_module":4,"doc_module_name":25,"category_name":117,"show_sort_weight":118,"slug":119},9,"Religion & Spirituality",20,"religion-spirituality",{"id":118,"doc_module":4,"doc_module_name":25,"category_name":121,"show_sort_weight":118,"slug":122},"World Cup","world-cup",{"id":55,"doc_module":4,"doc_module_name":25,"category_name":124,"show_sort_weight":55,"slug":125},"Lifestyle","lifestyle",{"id":127,"doc_module":4,"doc_module_name":25,"category_name":128,"show_sort_weight":97,"slug":129},19,"General","general",{"code":4,"msg":82,"data":131},{"doc_id":79,"user_id":132,"nickname":42,"user_avatar":133,"doc_module":4,"category_id":112,"category_name":29,"doc_title":10,"doc_description":12,"doc_content":134,"file_id":135,"file_url":136,"file_type":137,"file_size":138,"view_count":55,"is_deleted":4,"is_public":22,"is_downloadable":22,"audit_status":22,"page_count":139,"language":140,"language_code":8,"site_id":7,"html_lang":8,"table_of_contents":141,"faqs":142,"seo_title":143,"seo_description":12,"update_tm":144,"read_time":99,"preview_page_count":4},8796095462418,"https://ap-avatar.wpscdn.com/avatar/80000253c1241d02b47?x-image-process=image/resize,m_fixed,w_180,h_180&k=1778826106357471780","Integrating Topic Modeling and LLM Prompt Engineering into a Human-driven Approach to Analyze Interview  \nTranscripts  \nTeresa M. Ober ETS  \nPrinceton, NJ , USA [tober@ets.org](tober@ets.org)  \nKaryssa A. Courey Rice University Houston, TX, USA [kac23@rice.edu](kac23@rice.edu)  \nMichael Flor ETS  \nPrinceton, NJ , USA[mflor@ets.org](mflor@ets.org)  \nTopic modeling has become a widely used unsupervised machine learning method for extracting latent themes from large textual datasets. However, the interpretability of these themes often relies heavily on human judgment, which can limit transparency and reproducibility. Recent advances in large language models (LLMs) and prompt engineering offer new opportunities to enhance the interpretability and scalability of topic modeling outputs. This study presents a hybrid, human-in-the-loop methodological framework that integrates topic modeling, LLM prompting, and human-derived codes to support rigorous qualitative analysis. We apply this framework to focus group interviews with 13 U.S. teachers discussing the conceptualization and assessment of communication and digital literacy skills within competency-based education (CBE) contexts. The multi-stage process includes semantic clustering, LLM-assisted topic labeling, and iterative codebook refinement, enabling both scale and interpretive depth. Our findings demonstrate that this approach supports construct alignment, thematic stability, and methodological transparency, while preserving the contextual richness of qualitative data. We also highlight the importance of human oversight in guiding LLM outputsand ensuring theoretical coherence. This work contributes to emerging best practices for integrating AI tools into qualitative educational research by offering a replicable approach for analyzing complex, open-ended data that maintains both scalability and interpretability. The framework demonstrates how computational tools can augment human interpretive expertise while maintaining the epistemological integrity essential to qualitative inquiry. Supplemental materials are available at: [https://doi.org/10.17605/osf.io/4q6w8](https://doi.org/10.17605/osf.io/4q6w8) .  \nKeywords: large language models, topic modeling, qualitative analysis, skills assessment, human-AI  \n1. INTRODUCTION  \nThe integration of artificial intelligence (AI), particularly large language models (LLMs), into qualitative analysis has the apparent promise of enabling researchers to analyze large textual datasets and identify complex patterns reflecting variation in student learning with  \n156 Journal of Educational Data Mining, Volume 18, No 1, 2026  \nunprecedented speed and scale (see Schroeder et al., 2025) . One perceived advantage of using LLMs for qualitative data analysis is the promise of efficiency. Traditional approaches to qualitative analysis that rely on manual coding processes can be personnel and time-intensive and may produce codes that are subject to interpretive variability (Dai et al., 2023) . These constraints become particularly problematic when analyzing large amounts of open-ended qualitative data.  \nComputational methods, particularly hybrid human-machine learning approaches, have shown promise in enhancing the efficiency and reliability of qualitative coding without displacing the interpretive depth of human analysis. For instance, one study found that integrating models like BERT into iterative coding processes can significantly reduce manual effort while maintaining or even improving coding reliability (Li et al., 2021) . Digital tools and emerging technologies have also transformed qualitative research practices, enabling new forms of data collection, management, and analysis, though they introduce unique challenges that require careful methodological consideration (Magida, 2024) . While computational methods can amplify the extraction of information from rich qualitative data, the emergence of meaning remains rooted in interpretive processes rat","cbCaijT8aKzboeTW","https://ap.wps.com/l/cbCaijT8aKzboeTW","pdf",369792,24,"English","# INTRODUCTION\n## Hybrid human-machine approaches for qualitative analysis\n## Interpretability and transparency challenges of LLMs\n## Theme stability and epistemological integrity","[{\"question\":\"What problem does the study address in topic modeling for qualitative research?\",\"answer\":\"It addresses limited transparency and reproducibility when topic interpretability depends heavily on human judgment.\"},{\"question\":\"How does the proposed framework combine topic modeling, LLM prompting, and human input?\",\"answer\":\"It uses semantic clustering, LLM-assisted topic labeling, and iterative codebook refinement supported by human-derived codes.\"},{\"question\":\"What benefits do the findings report for using this human-in-the-loop approach?\",\"answer\":\"The approach supports construct alignment, thematic stability, and methodological transparency while preserving the contextual richness of qualitative data.\"}]","Integrating Topic Modeling and LLM Prompt Engineering into a Human-driven Approach to Analyze Interview Transcripts | PDF",1792034874]