[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"detail-sidebar-cat-0-en-105":3,"doc-seo-595592-105":59,"doc-detail-595592-en":130},{"code":4,"msg":5,"data":6},0,"success",[7,13,18,23,28,33,38,43,48,51,55],{"id":8,"doc_module":4,"doc_module_name":9,"category_name":10,"show_sort_weight":11,"slug":12},1,"Document","Story & Novel",90,"story-novel",{"id":14,"doc_module":4,"doc_module_name":9,"category_name":15,"show_sort_weight":16,"slug":17},2,"Literature",80,"literature",{"id":19,"doc_module":4,"doc_module_name":9,"category_name":20,"show_sort_weight":21,"slug":22},4,"Exam",70,"exam",{"id":24,"doc_module":4,"doc_module_name":9,"category_name":25,"show_sort_weight":26,"slug":27},5,"Comic",60,"comic",{"id":29,"doc_module":4,"doc_module_name":9,"category_name":30,"show_sort_weight":31,"slug":32},6,"Technology",50,"technology",{"id":34,"doc_module":4,"doc_module_name":9,"category_name":35,"show_sort_weight":36,"slug":37},7,"Healthcare",40,"healthcare",{"id":39,"doc_module":4,"doc_module_name":9,"category_name":40,"show_sort_weight":41,"slug":42},8,"Research & Report",30,"research-report",{"id":44,"doc_module":4,"doc_module_name":9,"category_name":45,"show_sort_weight":46,"slug":47},9,"Religion & Spirituality",20,"religion-spirituality",{"id":46,"doc_module":4,"doc_module_name":9,"category_name":49,"show_sort_weight":46,"slug":50},"World Cup","world-cup",{"id":52,"doc_module":4,"doc_module_name":9,"category_name":53,"show_sort_weight":52,"slug":54},10,"Lifestyle","lifestyle",{"id":56,"doc_module":4,"doc_module_name":9,"category_name":57,"show_sort_weight":24,"slug":58},19,"General","general",{"code":4,"msg":60,"data":61},"ok",{"site_id":62,"language":63,"slug":64,"title":65,"keywords":66,"description":67,"schema_data":68,"social_meta":123,"head_meta":125,"extra_data":127,"updated_unix":129},105,"en","a-systematic-assessment-of-none-of-the-above-on-multiple-choice-tests-in-a-first-year-psychology-classroom","A Systematic Assessment of 'None of the Above' on Multiple Choice Tests in a First Year Psychology Classroom","","Multiple-choice writing guidelines diverge on whether to use “none of the above” (NOTA), and prior empirical work reports mixed effects. Evidence has typically treated NOTA either as the correct option or as a distractor, evaluating changes in difficulty and discrimination. This study systematically replaced distractors with NOTA across versions of an introductory psychology exam to quantify distractor quality and estimate NOTA’s effects relative to what it replaced. NOTA increased difficulty only when it was correct and reduced discrimination when replacing the most effective distractors, supporting avoidance of NOTA in multiple-choice tests.",{"@graph":69,"@context":122},[70,84,105],{"@type":71,"itemListElement":72},"BreadcrumbList",[73,77,79,82],{"item":74,"name":75,"@type":76,"position":8},"https://docshare.wps.com","Home","ListItem",{"item":78,"name":9,"@type":76,"position":14},"https://docshare.wps.com/document/",{"item":80,"name":20,"@type":76,"position":81},"https://docshare.wps.com/document/exam/",3,{"item":83,"name":65,"@type":76,"position":19},"https://docshare.wps.com/document/a-systematic-assessment-of-none-of-the-above-on-multiple-choice-tests-in-a-first-year-psychology-classroom/595592/",{"url":83,"name":65,"@type":85,"image":86,"author":91,"headline":65,"publisher":94,"fileFormat":97,"inLanguage":63,"description":67,"dateModified":98,"datePublished":99,"encodingFormat":97,"isAccessibleForFree":100,"interactionStatistic":101},"DigitalDocument",{"url":87,"@type":88,"width":89,"height":90},"https://docshare.wps.com/thumbnails/a-systematic-assessment-of-none-of-the-above-on-multiple-choice-tests-in-a-first-year-psychology-classroom/595592.png","ImageObject",300,407,{"name":92,"@type":93},"Well Done","Person",{"url":74,"name":95,"@type":96},"DocShare","Organization","application/pdf","2026-10-21","2026-10-15",true,{"@type":102,"interactionType":103,"userInteractionCount":81},"InteractionCounter",{"@type":104},"ViewAction",{"@type":106,"mainEntity":107},"FAQPage",[108,114,118],{"name":109,"@type":110,"acceptedAnswer":111},"What problem does the study address about “none of the above” (NOTA)?","Question",{"text":112,"@type":113},"It examines how NOTA affects multiple-choice performance, specifically question difficulty and discrimination, and why results from earlier studies have been mixed.","Answer",{"name":115,"@type":110,"acceptedAnswer":116},"How did the authors test the impact of NOTA on exam questions?",{"text":117,"@type":113},"They systematically replaced each distractor with NOTA across different versions of an introductory psychology examination, enabling comparison against the quality of the distractor that was removed.",{"name":119,"@type":110,"acceptedAnswer":120},"What were the main results regarding difficulty and discrimination?",{"text":121,"@type":113},"NOTA increased question difficulty only when it served as the correct response, while it decreased discrimination when it replaced the most effective distractors; discrimination was unaffected when replacing the correct response or the lowest-quality distractor.","https://schema.org",{"og:url":83,"og:type":124,"og:title":65,"og:site_name":95,"og:description":67},"article",{"robots":126,"canonical":83},"index,follow",{"doc_id":128,"site_id":62},595592,1792461231,{"code":4,"msg":5,"data":131},{"doc_id":128,"user_id":132,"nickname":92,"user_avatar":133,"doc_module":4,"category_id":19,"category_name":20,"doc_title":65,"doc_description":67,"doc_content":134,"file_id":135,"file_url":136,"file_type":137,"file_size":138,"view_count":81,"is_deleted":4,"is_public":8,"is_downloadable":8,"audit_status":8,"page_count":139,"language":140,"language_code":63,"site_id":62,"html_lang":63,"table_of_contents":141,"faqs":142,"seo_title":143,"seo_description":67,"update_tm":144,"read_time":145},5909892210082,"https://ap-avatar.wpscdn.com/avatar/56001069da27c9f41e8?x-image-process=image/resize,m_fixed,w_180,h_180&k=1789614185077863061","The Canadian Journal for the Scholarship of Teaching and Learning  \n\n| Volume 6 | Issue 3 | Article 2 |\n| --- | --- |\n| 12-11-2015\u003Cbr>A Systematic Assessment of‘None of the Above’ on Multiple Choice Tests in a First Year Psychology Classroom\u003Cbr>Matthew V. Pachai\u003Cbr>McMaster University, [mattpachai@gmail.com](mattpachai@gmail.com)\u003Cbr>David DiBattista\u003Cbr>Brock University, [david.dibattista@brocku.ca](david.dibattista@brocku.ca)\u003Cbr>JosephA. Kim\u003Cbr>McMaster University, [kimjoe@mcmaster.ca](kimjoe@mcmaster.ca)\u003Cbr>Follow this and additional works at: [http://ir.lib.uwo.ca/cjsotl_rcacea](http://ir.lib.uwo.ca/cjsotl_rcacea)\u003Cbr> Part of the Educational Assessment, Evaluation, and Research Commons, Educational Methods Commons, Scholarship of Teaching and Learning Commons, and the Teacher Education and Professional Development Commons |  |\n\nRecommended Citation  \nPachai, Matthew V.; DiBattista, David; and Kim, JosephA. (2015) \"A Systematic Assessment of‘None of the Above’ on Multiple Choice Tests in a First Year Psychology Classroom,\" The Canadian Journal for the Scholarship of Teaching and Learning: Vol. 6: Iss. 3, Article 2.  \nAvailable at: [http://ir.lib.uwo.ca/cjsotl_rcacea/vol6/iss3/2](http://ir.lib.uwo.ca/cjsotl_rcacea/vol6/iss3/2)  \nA Systematic Assessment of‘None of the Above’ on Multiple Choice Tests in a First Year Psychology Classroom  \nAbstract  \nMultiple choice writing guidelines are decidedly split on the use of ‘none of the above’ (NOTA), with some authors discouraging and others advocating its use. Moreover, empirical studies of NOTA have produced mixed results. Generally, these studies have utilized NOTA as either the correct response or a distractor and assessed its effect on difficulty and discrimination. In these studies, NOTA commonly yields increased difficulty when it is used as the correct response, and no change in discrimination regardless of usage.  \nHowever, when NOTA is implemented as a distractor, rarely is consideration given to the distractor that could have been written in its place. Here, we systematically replaced each distractor in a series of questions with NOTA across different versions of an Introductory Psychology examination. This approach allowed us to quantify the quality of each distractor based on its relative discrimination index and assess the effect of NOTA relative to the quality of distractor it replaced. Moreover, our use of large Introductory Psychology examinations afforded highly stable difficulty and discrimination estimates. We found that NOTA increased question difficulty only when it was the correct response, with no effect on difficulty of replacing any distractor type with NOTA. Moreover, we found that NOTA decreased discrimination when it replaced the most effective distractors, with no effect on discrimination of replacing either the correct response or lowest quality distractor with NOTA. These results replicate the common finding that inclusion of NOTA as the correct response increases question difficulty by equally luring high-performing and low-performing students toward distractors. Moreover, we have shown that including NOTA as a distractor can reduce discrimination if used in lieu of a well written alternative, suggesting that multiple choice authors should avoid using NOTA on multiple choice tests.  \nLes guides de rédaction pour les questions à choix multiple sont incontestablement partagés sur l’usage de laréponse « aucune des situations ci-dessus ». Certains auteurs déconseillent de l’employer alors que d’autres préconisent de le faire. De plus, des études empiriques de l’emploi de cette expression ont mené à des résultats mitigés. En général, ces études ont utilisé l’option « aucune des situations ci-dessus » soit comme étant laréponse correcte soit comme distracteur et ont évalué ses effets sur la difficulté et le discernement. Dans ces études, la réponse « aucune des situations ci-dessus » mène généralement à une augmentation de la difficulté quand l’option est em","cbCaiep6oJ0G1yf6","https://ap.wps.com/l/cbCaiep6oJ0G1yf6","pdf",599383,17,"English","# Abstract\n## Research approach\n## Key findings\n## Implications","[{\"question\":\"What problem does the study address about “none of the above” (NOTA)?\",\"answer\":\"It examines how NOTA affects multiple-choice performance, specifically question difficulty and discrimination, and why results from earlier studies have been mixed.\"},{\"question\":\"How did the authors test the impact of NOTA on exam questions?\",\"answer\":\"They systematically replaced each distractor with NOTA across different versions of an introductory psychology examination, enabling comparison against the quality of the distractor that was removed.\"},{\"question\":\"What were the main results regarding difficulty and discrimination?\",\"answer\":\"NOTA increased question difficulty only when it served as the correct response, while it decreased discrimination when it replaced the most effective distractors; discrimination was unaffected when replacing the correct response or the lowest-quality distractor.\"}]","A Systematic Assessment of 'None of the Above' on Multiple Choice Tests in a First Year Psychology Classroom | PDF",1792081441,43]