[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"detail-sidebar-cat-0-en-105":3,"doc-seo-577739-105":59,"doc-detail-577739-en":130},{"code":4,"msg":5,"data":6},0,"success",[7,13,18,23,28,33,38,43,48,51,55],{"id":8,"doc_module":4,"doc_module_name":9,"category_name":10,"show_sort_weight":11,"slug":12},1,"Document","Story & Novel",90,"story-novel",{"id":14,"doc_module":4,"doc_module_name":9,"category_name":15,"show_sort_weight":16,"slug":17},2,"Literature",80,"literature",{"id":19,"doc_module":4,"doc_module_name":9,"category_name":20,"show_sort_weight":21,"slug":22},4,"Exam",70,"exam",{"id":24,"doc_module":4,"doc_module_name":9,"category_name":25,"show_sort_weight":26,"slug":27},5,"Comic",60,"comic",{"id":29,"doc_module":4,"doc_module_name":9,"category_name":30,"show_sort_weight":31,"slug":32},6,"Technology",50,"technology",{"id":34,"doc_module":4,"doc_module_name":9,"category_name":35,"show_sort_weight":36,"slug":37},7,"Healthcare",40,"healthcare",{"id":39,"doc_module":4,"doc_module_name":9,"category_name":40,"show_sort_weight":41,"slug":42},8,"Research & Report",30,"research-report",{"id":44,"doc_module":4,"doc_module_name":9,"category_name":45,"show_sort_weight":46,"slug":47},9,"Religion & Spirituality",20,"religion-spirituality",{"id":46,"doc_module":4,"doc_module_name":9,"category_name":49,"show_sort_weight":46,"slug":50},"World Cup","world-cup",{"id":52,"doc_module":4,"doc_module_name":9,"category_name":53,"show_sort_weight":52,"slug":54},10,"Lifestyle","lifestyle",{"id":56,"doc_module":4,"doc_module_name":9,"category_name":57,"show_sort_weight":24,"slug":58},19,"General","general",{"code":4,"msg":60,"data":61},"ok",{"site_id":62,"language":63,"slug":64,"title":65,"keywords":66,"description":67,"schema_data":68,"social_meta":123,"head_meta":125,"extra_data":127,"updated_unix":129},105,"en","what-makes-a-killer-question-killer-a-text-mining-analysis-of-high-difficulty-questions-in-the-korean-csat-english-section","What Makes a Killer Question Killer? - A Text Mining Analysis of High-difficulty Questions in the Korean CSAT English Section","","Investigates the thematic and lexical characteristics of high-difficulty English reading items (“killer questions”) in the Korean College Scholastic Ability Test (CSAT) from 2018 to 2025. Applies text mining, including Latent Dirichlet Allocation (LDA) and CEFR-based lexical profiling, to identify patterns in topics and vocabulary levels. Results show these items disproportionately draw on abstract, academic themes and use substantial C1+ vocabulary, raising concerns about construct validity and fairness by reflecting background knowledge or test-taking strategies rather than reading proficiency. Calls for evidence-based item development aligned with curriculum goals.",{"@graph":69,"@context":122},[70,84,105],{"@type":71,"itemListElement":72},"BreadcrumbList",[73,77,79,82],{"item":74,"name":75,"@type":76,"position":8},"https://docshare.wps.com","Home","ListItem",{"item":78,"name":9,"@type":76,"position":14},"https://docshare.wps.com/document/",{"item":80,"name":40,"@type":76,"position":81},"https://docshare.wps.com/document/research-report/",3,{"item":83,"name":65,"@type":76,"position":19},"https://docshare.wps.com/document/what-makes-a-killer-question-killer-a-text-mining-analysis-of-high-difficulty-questions-in-the-korean-csat-english-section/577739/",{"url":83,"name":65,"@type":85,"image":86,"author":91,"headline":65,"publisher":94,"fileFormat":97,"inLanguage":63,"description":67,"dateModified":98,"datePublished":99,"encodingFormat":97,"isAccessibleForFree":100,"interactionStatistic":101},"DigitalDocument",{"url":87,"@type":88,"width":89,"height":90},"https://docshare.wps.com/thumbnails/what-makes-a-killer-question-killer-a-text-mining-analysis-of-high-difficulty-questions-in-the-korean-csat-english-section/577739.png","ImageObject",300,407,{"name":92,"@type":93},"Olivia Brown","Person",{"url":74,"name":95,"@type":96},"DocShare","Organization","application/pdf","2026-11-16","2026-10-15",true,{"@type":102,"interactionType":103,"userInteractionCount":52},"InteractionCounter",{"@type":104},"ViewAction",{"@type":106,"mainEntity":107},"FAQPage",[108,114,118],{"name":109,"@type":110,"acceptedAnswer":111},"What are “killer questions” in the Korean CSAT English section?","Question",{"text":112,"@type":113},"They refer to high-difficulty reading items perceived as very challenging, even for top-performing students, and often requiring higher-order reasoning from academic passages.","Answer",{"name":115,"@type":110,"acceptedAnswer":116},"Which methods does the study use to analyze high-difficulty items?",{"text":117,"@type":113},"It uses text mining methods, specifically Latent Dirichlet Allocation (LDA) for topic modeling and CEFR-based lexical profiling for vocabulary analysis.",{"name":119,"@type":110,"acceptedAnswer":120},"What concerns does the study raise about the fairness and validity of killer questions?",{"text":121,"@type":113},"The findings suggest misalignment with curriculum goals and indicate that the items’ abstract themes and advanced vocabulary may test background knowledge or test-taking strategies instead of genuine reading proficiency.","https://schema.org",{"og:url":83,"og:type":124,"og:title":65,"og:site_name":95,"og:description":67},"article",{"robots":126,"canonical":83},"index,follow",{"doc_id":128,"site_id":62},577739,1792208596,{"code":4,"msg":5,"data":131},{"doc_id":128,"user_id":132,"nickname":92,"user_avatar":133,"doc_module":4,"category_id":39,"category_name":40,"doc_title":65,"doc_description":67,"doc_content":134,"file_id":135,"file_url":136,"file_type":137,"file_size":138,"view_count":52,"is_deleted":4,"is_public":8,"is_downloadable":8,"audit_status":8,"page_count":46,"language":139,"language_code":63,"site_id":62,"html_lang":63,"table_of_contents":140,"faqs":141,"seo_title":142,"seo_description":67,"update_tm":143,"read_time":31,"preview_page_count":4},16904993612988,"https://ap-avatar.wpscdn.com/davatar_a8503ba1806abce46bf441b54a3ca4cd","DOI: [https://doi.org/10.15858/engtea.80.3.202509.3](https://doi.org/10.15858/engtea.80.3.202509.3)  \n[https://journal.kate.or.kr](https://journal.kate.or.kr)  \nWhat Makes a Killer Question Killer? A Text Mining Analysis of High-difficulty Questions in the  \nKorean CSAT English Section*  \nJeong-eun Kim**  \nKim, Jeong-eun. (2025). What makes a killer question killer? A text mining analysis of high-difficulty questions in the Korean CSAT English section. English Teaching, 80(3), 3–22.  \nThis study investigated the thematic and lexical characteristics of high-difficulty English reading items—commonly referred to as killer questions—in the Korean College Scholastic Ability Test (CSAT) between 2018 and 2025. Using text mining methods, including Latent Dirichlet Allocation (LDA) and CEFR-based lexical profiling, the analysis reveals that these items disproportionately addressed abstract, academic topics such as cultural heritage, moral dilemmas, and cognitive science.  \nLexical analysis further indicated that over 30% of the words used are at the C1 level or above, reflecting substantial vocabulary demands. These findings point to amisalignment with the national curriculum, which is designed to target B1–B2 proficiency with more familiar and accessible topics. The combined challenge of abstract themes and advanced vocabulary raises concerns about the construct validity and fairness of these items, suggesting that they may measure background knowledge or test-taking strategies rather than genuine English reading proficiency. The study concludes by calling for evidence-based item development and closer alignment with curricular goals to ensure valid and equitable high-stakes language assessment.  \nKey words: CSAT English, killer questions, text mining, topic modeling, lexical profiling, test validity  \n*I sincerely thank my undergraduate research assistants, Yunseo Jang, YoonJung Park, and Yunseo Park, for their dedicated support with data work and their eagerness to learn new skills such as Python and algorithmic thinking. Their contributions were essential to this project’s success.  \n**Author: Jeong-eun Kim, Associate Professor, Department of English Language & Literature, Jeonbuk National University; 567 Baekje-daero, Deokjin-gu, Jeonju-si, Jeollabuk-do 54896, Korea; Email: [jek48@jbnu.ac.kr](jek48@jbnu.ac.kr)  \nReceived 30 June 2025; Reviewed 25 July 2025; Accepted 23 September 2025  \n© 2025 The Korea Association of Teachers of English (KATE)  \n1. INTRODUCTION  \nIn South Korea, the College Scholastic Ability Test (CSAT) serves as a high-stakes, centralized gateway to higher education. Since the adoption of a criterion-referenced grading system in 2018, the proportion of students earning the top grade (Level 1) in the CSAT English section has steadily declined—from 10.03% in 2018 to 6.25% in 2022, and further to 4.71% in 2024 (S.-H. Kim, 2024) . At the center of this decline are the so-called killer questions—items perceived to be very challenging that even high-performing students struggle to answer them. These items, often demanding higher-order reasoning based on academic or technical passages, significantly affect score distributions (Kim & Lee, 2025) .  \nConcerns have been raised about the misalignment between these high-difficulty items and the national curriculum (Jang, 2023) . While the 2015 Revised English Curriculum emphasizes accessible content and B1-level reading proficiency (Ministry of Education, 2015; Choi, Lee, Oh, & So, 2022), empirical studies have shown that many actual test items, especially killer questions, feature dense academic discourse and advanced vocabulary beyond curriculum expectations (Y. Choi, 2023; Na & Lee, 2024) . For instance, the 2023 CSAT English item Q34 was criticized for its abstract content and complex argumentative structure (Ministry of Education, 2023; see Appendix A). These suggest that killer questions may exceed curriculum-aligned difficulty and impose high cognitive demands.  \nAlth","cbCaiv6yIywwcDFR","https://ap.wps.com/l/cbCaiv6yIywwcDFR","pdf",597943,"English","# 1. Introduction\n## Declining top-grade performance and killer questions\n## Curriculum misalignment concerns\n# 2. Literature Review\n## Validity issues in killer questions of the CSAT English section\n## Construct validity studies","[{\"question\":\"What are “killer questions” in the Korean CSAT English section?\",\"answer\":\"They refer to high-difficulty reading items perceived as very challenging, even for top-performing students, and often requiring higher-order reasoning from academic passages.\"},{\"question\":\"Which methods does the study use to analyze high-difficulty items?\",\"answer\":\"It uses text mining methods, specifically Latent Dirichlet Allocation (LDA) for topic modeling and CEFR-based lexical profiling for vocabulary analysis.\"},{\"question\":\"What concerns does the study raise about the fairness and validity of killer questions?\",\"answer\":\"The findings suggest misalignment with curriculum goals and indicate that the items’ abstract themes and advanced vocabulary may test background knowledge or test-taking strategies instead of genuine reading proficiency.\"}]","What Makes a Killer Question Killer? - A Text Mining Analysis of High-difficulty Questions in the Korean CSAT English Section | PDF",1792032323]