[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"detail-sidebar-cat-0-en-105":3,"doc-seo-580741-105":59,"doc-detail-580741-en":130},{"code":4,"msg":5,"data":6},0,"success",[7,13,18,23,28,33,38,43,48,51,55],{"id":8,"doc_module":4,"doc_module_name":9,"category_name":10,"show_sort_weight":11,"slug":12},1,"Document","Story & Novel",90,"story-novel",{"id":14,"doc_module":4,"doc_module_name":9,"category_name":15,"show_sort_weight":16,"slug":17},2,"Literature",80,"literature",{"id":19,"doc_module":4,"doc_module_name":9,"category_name":20,"show_sort_weight":21,"slug":22},4,"Exam",70,"exam",{"id":24,"doc_module":4,"doc_module_name":9,"category_name":25,"show_sort_weight":26,"slug":27},5,"Comic",60,"comic",{"id":29,"doc_module":4,"doc_module_name":9,"category_name":30,"show_sort_weight":31,"slug":32},6,"Technology",50,"technology",{"id":34,"doc_module":4,"doc_module_name":9,"category_name":35,"show_sort_weight":36,"slug":37},7,"Healthcare",40,"healthcare",{"id":39,"doc_module":4,"doc_module_name":9,"category_name":40,"show_sort_weight":41,"slug":42},8,"Research & Report",30,"research-report",{"id":44,"doc_module":4,"doc_module_name":9,"category_name":45,"show_sort_weight":46,"slug":47},9,"Religion & Spirituality",20,"religion-spirituality",{"id":46,"doc_module":4,"doc_module_name":9,"category_name":49,"show_sort_weight":46,"slug":50},"World Cup","world-cup",{"id":52,"doc_module":4,"doc_module_name":9,"category_name":53,"show_sort_weight":52,"slug":54},10,"Lifestyle","lifestyle",{"id":56,"doc_module":4,"doc_module_name":9,"category_name":57,"show_sort_weight":24,"slug":58},19,"General","general",{"code":4,"msg":60,"data":61},"ok",{"site_id":62,"language":63,"slug":64,"title":65,"keywords":66,"description":67,"schema_data":68,"social_meta":123,"head_meta":125,"extra_data":127,"updated_unix":129},105,"en","research-matters-issue-39-comparative-judgement-exercise-in-grade-boundaries","Research Matters - Issue 39 - Comparative judgement exercise in grade boundaries","","This research investigates how comparative judgement (CJ) influences qualification-level grade boundary setting when evidence comes from different exam papers. Using evidence from three qualifications in autumn 2020, where a separate CJ exercise was completed for each paper to preserve standards, the study tests counterfactual scenarios using only one paper’s CJ results. It examines consistency in estimated cohort ability changes across series, and the effect of single-paper evidence on the precision of identifying appropriate grade boundaries.",{"@graph":69,"@context":122},[70,84,105],{"@type":71,"itemListElement":72},"BreadcrumbList",[73,77,79,82],{"item":74,"name":75,"@type":76,"position":8},"https://docshare.wps.com","Home","ListItem",{"item":78,"name":9,"@type":76,"position":14},"https://docshare.wps.com/document/",{"item":80,"name":40,"@type":76,"position":81},"https://docshare.wps.com/document/research-report/",3,{"item":83,"name":65,"@type":76,"position":19},"https://docshare.wps.com/document/research-matters-issue-39-comparative-judgement-exercise-in-grade-boundaries/580741/",{"url":83,"name":65,"@type":85,"image":86,"author":91,"headline":65,"publisher":94,"fileFormat":97,"inLanguage":63,"description":67,"dateModified":98,"datePublished":99,"encodingFormat":97,"isAccessibleForFree":100,"interactionStatistic":101},"DigitalDocument",{"url":87,"@type":88,"width":89,"height":90},"https://docshare.wps.com/thumbnails/research-matters-issue-39-comparative-judgement-exercise-in-grade-boundaries/580741.png","ImageObject",300,407,{"name":92,"@type":93},"Riley","Person",{"url":74,"name":95,"@type":96},"DocShare","Organization","application/pdf","2026-11-16","2026-10-15",true,{"@type":102,"interactionType":103,"userInteractionCount":44},"InteractionCounter",{"@type":104},"ViewAction",{"@type":106,"mainEntity":107},"FAQPage",[108,114,118],{"name":109,"@type":110,"acceptedAnswer":111},"What is the main research question about comparative judgement and grade boundaries?","Question",{"text":112,"@type":113},"The article asks whether using comparative judgement from only one exam paper is sufficient to set qualification-level grade boundaries at the appropriate standard.","Answer",{"name":115,"@type":110,"acceptedAnswer":116},"How is comparative judgement (CJ) typically used in awarding?",{"text":117,"@type":113},"Judges compare many pairs of scripts, selecting which script has superior overall performance. Decisions are then analyzed to identify score pairs across current and previous series that are equivalent under expert judgement.",{"name":119,"@type":110,"acceptedAnswer":120},"What changes when CJ evidence comes from only one paper?",{"text":121,"@type":113},"The study evaluates whether evidence from different papers gives a consistent picture of cohort ability changes, and how relying on a single paper affects the precision of identifying suitable qualification-level grade boundaries.","https://schema.org",{"og:url":83,"og:type":124,"og:title":65,"og:site_name":95,"og:description":67},"article",{"robots":126,"canonical":83},"index,follow",{"doc_id":128,"site_id":62},580741,1792519147,{"code":4,"msg":5,"data":131},{"doc_id":128,"user_id":132,"nickname":92,"user_avatar":133,"doc_module":4,"category_id":39,"category_name":40,"doc_title":65,"doc_description":67,"doc_content":134,"file_id":135,"file_url":136,"file_type":137,"file_size":138,"view_count":44,"is_deleted":4,"is_public":8,"is_downloadable":8,"audit_status":8,"page_count":139,"language":140,"language_code":63,"site_id":62,"html_lang":63,"table_of_contents":141,"faqs":142,"seo_title":143,"seo_description":67,"update_tm":144,"read_time":145,"preview_page_count":4},1374391975076,"https://ap-avatar.wpscdn.com/avatar/14000253ca4ec9f6853?x-image-process=image/resize,m_fixed,w_180,h_180&k=1783305029341752051","Research Matters / 39  \nA Cambridge University Press & Assessment publication  \nISSN: 1755-6031  \nJournal homepage: [https://www.cambridgeassessment.org.uk/our-research/all](https://www.cambridgeassessment.org.uk/our-research/all)published-resources/research-matters/  \nIs one comparative judgement exercise for one exam paper sufficient to set qualification-level grade boundaries?  \nTom Benton  \nTo cite this article: Benton, T. (2025). Is one comparative judgement exercise for one exam paper sufficient to set qualification-level grade boundaries? Research Matters: A Cambridge University Press & Assessment publication, 39, 26–38.  \n[https://doi.org/10.17863/CAM.116168](https://doi.org/10.17863/CAM.116168)  \nTo link this article: [https://www.cambridgeassessment.org.uk/Images/research](https://www.cambridgeassessment.org.uk/Images/research)matters-39-is-one-comparative-judgement-exercise-for-one-exam-papersufficient-to-set-qualification-level-grade-boundaries.pdf  \nAbstract:  \nThis research draws on evidence from three qualifications taken in autumn 2020, when comparative judgement (CJ) was used as a key source of data in setting grade boundaries. In these cases, a separate CJ exercise was completed for each individual paper in the qualification so that standards could be maintained from a previous series. In this article, we explore what would have happened had werelied on a single CJ exercise on one paper to maintain standards in the whole qualification. We first examine whether evidence from different papers provides a consistent picture of changes in cohort ability between series. We then explore the impact of relying on evidence from one paper only on the precision with which we can identify appropriate qualification-level grade boundaries using CJ.  \nCambridge University Press & Assessment is committed to making its documents accessible in accordance with the WCAG 2.1 Standard. We’re always looking to improve the accessibility of our documents. If you find any problems or you think we’re not meeting accessibility requirements, contact our team:  \nResearch Division, [ResearchDivision@cambridge.org](ResearchDivision@cambridge.org)  \nIf you need this document in a different format contact us, telling us your name, email address and requirements and we will respond within 15 working days.  \n© Cambridge University Press & Assessment 2025  \nFull Terms & Conditions of access and use can be found at  \nT&C: Terms and Conditions | Cambridge University Press & Assessment  \n© Cambridge University Press & Assessment 2025  \nIs one comparative judgement exercise for one exam paper sufficient to set qualification-level grade boundaries?  \nTom Benton (Research Division)  \nIntroduction  \nIn high-stakes qualifications such as GCSEs and A Levels in England, grade boundaries refer to the minimum score a candidate requires to achieve a given grade. Historically, the way in which these have been determined has always included an element of human expert judgement (Benton & Elliott, 2015). Most commonly this has been in the form of expert examiners in the relevant subjects reviewing examination scripts with scores close to suggested grade boundaries and indicating whether or not they feel they are of sufficient quality to be awarded the focus grade. However, for some time, a number of authors have argued that comparative judgement (CJ) may provide a more effective means of incorporating inputs from subject experts into the process of determining grade boundaries (e.g., Bramley, 2007; Curcin et al., 2019). The awarding body OCR conducted a substantial programme of research investigating this technique, which is described in Benton et al. (2022).  \nIn the context of awarding, a typical CJ exercise involves judges (usually examiners) comparing many pairs of scripts. Each pair consists of one script from the current exam series and one from a previous series. Note that the scripts from different series will consist of responses to different sets of que","cbCaijhM37AcMQKd","https://ap.wps.com/l/cbCaijhM37AcMQKd","pdf",376386,14,"English","# Introduction\n## Comparative judgement in awarding\n## One CJ exercise per paper vs one paper only","[{\"question\":\"What is the main research question about comparative judgement and grade boundaries?\",\"answer\":\"The article asks whether using comparative judgement from only one exam paper is sufficient to set qualification-level grade boundaries at the appropriate standard.\"},{\"question\":\"How is comparative judgement (CJ) typically used in awarding?\",\"answer\":\"Judges compare many pairs of scripts, selecting which script has superior overall performance. Decisions are then analyzed to identify score pairs across current and previous series that are equivalent under expert judgement.\"},{\"question\":\"What changes when CJ evidence comes from only one paper?\",\"answer\":\"The study evaluates whether evidence from different papers gives a consistent picture of cohort ability changes, and how relying on a single paper affects the precision of identifying suitable qualification-level grade boundaries.\"}]","Research Matters - Issue 39 - Comparative judgement exercise in grade boundaries | PDF",1792038207,35]