[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"detail-sidebar-cat-0-en-105":3,"doc-seo-158063-105":59,"doc-detail-158063-en":131},{"code":4,"msg":5,"data":6},0,"success",[7,13,18,23,28,33,38,43,48,51,55],{"id":8,"doc_module":4,"doc_module_name":9,"category_name":10,"show_sort_weight":11,"slug":12},1,"Document","Story & Novel",90,"story-novel",{"id":14,"doc_module":4,"doc_module_name":9,"category_name":15,"show_sort_weight":16,"slug":17},2,"Literature",80,"literature",{"id":19,"doc_module":4,"doc_module_name":9,"category_name":20,"show_sort_weight":21,"slug":22},4,"Exam",70,"exam",{"id":24,"doc_module":4,"doc_module_name":9,"category_name":25,"show_sort_weight":26,"slug":27},5,"Comic",60,"comic",{"id":29,"doc_module":4,"doc_module_name":9,"category_name":30,"show_sort_weight":31,"slug":32},6,"Technology",50,"technology",{"id":34,"doc_module":4,"doc_module_name":9,"category_name":35,"show_sort_weight":36,"slug":37},7,"Healthcare",40,"healthcare",{"id":39,"doc_module":4,"doc_module_name":9,"category_name":40,"show_sort_weight":41,"slug":42},8,"Research & Report",30,"research-report",{"id":44,"doc_module":4,"doc_module_name":9,"category_name":45,"show_sort_weight":46,"slug":47},9,"Religion & Spirituality",20,"religion-spirituality",{"id":46,"doc_module":4,"doc_module_name":9,"category_name":49,"show_sort_weight":46,"slug":50},"World Cup","world-cup",{"id":52,"doc_module":4,"doc_module_name":9,"category_name":53,"show_sort_weight":52,"slug":54},10,"Lifestyle","lifestyle",{"id":56,"doc_module":4,"doc_module_name":9,"category_name":57,"show_sort_weight":24,"slug":58},19,"General","general",{"code":4,"msg":60,"data":61},"ok",{"site_id":62,"language":63,"slug":64,"title":65,"keywords":66,"description":67,"schema_data":68,"social_meta":124,"head_meta":126,"extra_data":128,"updated_unix":130},105,"en","psychological-test-and-assessment-modeling-automated-distractor-generation-for-fill-in-the-blank-items","Psychological Test and Assessment Modeling - Automated Distractor Generation for Fill-in-the-Blank Items","","Large and continuous supplies of new items are increasingly demanded in language testing, making automated item generation (AIG) crucial for improving development efficiency. Multiple-choice item quality depends on effective distractors—incorrect but attractive options—so the work proposes a prompt-based learning approach for automatically generating distractors for fill-in-the-blank vocabulary items. The method reframes distractor generation as natural language generation and fine-tunes a transformer-based pretrained language model using prompt-based learning to support small-sample training. Effects of different prompts are examined and compared with a rule-based baseline using a standardized English proficiency assessment item type.",{"@graph":69,"@context":123},[70,84,106],{"@type":71,"itemListElement":72},"BreadcrumbList",[73,77,79,82],{"item":74,"name":75,"@type":76,"position":8},"https://docshare.wps.com","Home","ListItem",{"item":78,"name":9,"@type":76,"position":14},"https://docshare.wps.com/document/",{"item":80,"name":40,"@type":76,"position":81},"https://docshare.wps.com/document/research-report/",3,{"item":83,"name":65,"@type":76,"position":19},"https://docshare.wps.com/document/psychological-test-and-assessment-modeling-automated-distractor-generation-for-fill-in-the-blank-items/158063/",{"url":83,"name":65,"@type":85,"image":86,"author":91,"headline":65,"publisher":94,"fileFormat":97,"inLanguage":63,"description":67,"dateModified":98,"datePublished":99,"encodingFormat":97,"isAccessibleForFree":100,"interactionStatistic":101},"DigitalDocument",{"url":87,"@type":88,"width":89,"height":90},"https://docshare.wps.com/thumbnails/psychological-test-and-assessment-modeling-automated-distractor-generation-for-fill-in-the-blank-items/158063.png","ImageObject",300,407,{"name":92,"@type":93},"Đào","Person",{"url":74,"name":95,"@type":96},"DocShare","Organization","application/pdf","2026-10-31","2026-08-29",true,{"@type":102,"interactionType":103,"userInteractionCount":105},"InteractionCounter",{"@type":104},"ViewAction",11,{"@type":107,"mainEntity":108},"FAQPage",[109,115,119],{"name":110,"@type":111,"acceptedAnswer":112},"What problem does the paper address in language testing item development?","Question",{"text":113,"@type":114},"It addresses the need for large, continuous supplies of new test items and the difficulty of writing effective distractors for multiple-choice vocabulary questions.","Answer",{"name":116,"@type":111,"acceptedAnswer":117},"How does the proposed approach generate distractors for fill-in-the-blank items?",{"text":118,"@type":114},"It treats distractor generation as a natural language generation task and fine-tunes a transformer-based pretrained language model using prompt-based learning.",{"name":120,"@type":111,"acceptedAnswer":121},"Why is prompt-based learning emphasized in the fine-tuning process?",{"text":122,"@type":114},"Because testing programs often cannot provide the tens of thousands of training examples typically needed, and prompt-based learning is effective in small-sample scenarios.","https://schema.org",{"og:url":83,"og:type":125,"og:title":65,"og:site_name":95,"og:description":67},"article",{"robots":127,"canonical":83},"index,follow",{"doc_id":129,"site_id":62},158063,1787994248,{"code":4,"msg":5,"data":132},{"doc_id":129,"user_id":133,"nickname":92,"user_avatar":134,"doc_module":4,"category_id":39,"category_name":40,"doc_title":65,"doc_description":67,"doc_content":135,"file_id":136,"file_url":137,"file_type":138,"file_size":139,"view_count":105,"is_deleted":4,"is_public":8,"is_downloadable":8,"audit_status":8,"page_count":140,"language":141,"language_code":63,"site_id":62,"html_lang":63,"table_of_contents":142,"faqs":143,"seo_title":144,"seo_description":67,"update_tm":130,"read_time":145,"preview_page_count":4},1374402968488,"https://ap-avatar.wpscdn.com/davatar_29158cc5080c5b710cf443261637dec0","Psychological Test and Assessment Modeling, Volume 65, 2023 (1), 55-75  \nAutomated Distractor Generation for Fillin-the-Blank Items  \nUsing a Prompt-Based Learning Approach  \nJiyun Zu, Ikkyu Choi, Jiangang Hao  \nEducational Testing Service, 660 Rosedale Road, Princeton, NJ, 08540  \nAbstract  \nThere are heavy demands for large and continuous supplies of new items in language testing. Automated item generation (AIG), in which computerized algorithms are used to create test items, can potentially increase the efficiency of new item development to serve this demand. A challenge for multiple-choice items is to write effective distractors, that is, incorrect yet attractive (Haladyna, 2004) . We propose a prompt-based learning approach (Liu et al., 2021) for automatically generating distractors for one of the most common language-assessment item types, fill-in-the-blank vocabulary items. The proposed method treats distractor generation as a natural language generation task and utilizes a transformer-based, pretrained language model (Radford et al., 2019) fine-tuned to ensure appropriate and useful output. The fine-tuning process adopted a prompt-based learning approach, which has been found to be particularly effective in small-sample scenarios (Gao et al., 2021) . We illustrate this approach on a specific item type from a standardized English language proficiency assessment. Specifically, we study the effects of different prompts and demonstrate the effectiveness of the proposed prompt-based learning approach by comparing features of generated distractors with those from a rule-based approach.  \nKeywords: Automated distractor generation, automated item generation, natural language processing, deep learning language models, prompt-based learning  \nAuthor Note  \nCorrespondence concerning this article should be addressed to Jiyun Zu, Educational Testing  \nService, 660 Rosedale Road, Princeton, NJ, 08540. Email: [j](jzu@ets.org)[zu@ets.org](jzu@ets.org)  \nLanguage testing programs, like many other educational and psychological testing programs, face increasing demands for flexible test administrations. Since the COVID-19 pandemic, many language proficiency tests are offered tobe taken at home with more available testing dates. Example tests include high-stake tests such as the Duolingo English Test, Pearson PTE Academic, and the TOEFL iBT® test. This move towards flexible administrations has led to heavy demands for large and continuous supplies of new items.  \nAutomated item generation (AIG), in which computerized algorithms are used to create test items, can potentially increase the efficiency of new item development. For cognitive tests, a three-step template-based approach (Gierl & Lai, 2015) has been successfully used to automatically generate a variety of item types including mathematics achievement items (Embretson & Kingston, 2018) , medical knowledge items (Gierl et al., 2012), and fluid reasoning items (Kyllonen et al., 2019) . This approach involves developing item models that consist of components, creating content variants for each component, and then using computer algorithms to select and combine components into new items. The number of items created from one item model can be large because it is the product of the number of variants for each component. More recently, deep learning language models have been utilized for AIG for personality assessments, where the template-based approach is not suitable. For example, von Davier (2018) trained a long-short-term memory- (LSTM-) based recurrent neural network model and Hommel et al. (2022) fine-tuned a transformer-based deep learning language model to automatically generate personality statements (e.g., “I work hard”) .  \nLike personality assessments, AIG for language assessments will be influenced by advancement in natural language processing (NLP) technology. Over the last a few years, there have been several breakthroughs in the field of NLP. A prominent example is the use ","cbCaivflpdJfDXWo","https://ap.wps.com/l/cbCaivflpdJfDXWo","pdf",696086,21,"English","# Abstract\n# Introduction and Motivation\n## Automated item generation and distractor quality\n## NLP advances enabling AIG\n# Proposed Method\n## Prompt-based learning for small-sample fine-tuning\n## Fine-tuning transformer language models for vocabulary distractors\n# Evaluation Design\n## Studying prompt effects and comparing baselines","[{\"question\":\"What problem does the paper address in language testing item development?\",\"answer\":\"It addresses the need for large, continuous supplies of new test items and the difficulty of writing effective distractors for multiple-choice vocabulary questions.\"},{\"question\":\"How does the proposed approach generate distractors for fill-in-the-blank items?\",\"answer\":\"It treats distractor generation as a natural language generation task and fine-tunes a transformer-based pretrained language model using prompt-based learning.\"},{\"question\":\"Why is prompt-based learning emphasized in the fine-tuning process?\",\"answer\":\"Because testing programs often cannot provide the tens of thousands of training examples typically needed, and prompt-based learning is effective in small-sample scenarios.\"}]","Psychological Test and Assessment Modeling - Automated Distractor Generation for Fill-in-the-Blank Items | PDF",53]