Standard setting OSCE - A comparison of arbitrary and Hofstee methods in a low stake OSCE

Sep 20, 2026

Objectives compare cut scores and pass/fail rates produced by the arbitrary 60% method versus the Hofstee method in a year 4 end-of-semester OSCE, and evaluate whether the Hofstee approach can be used for future examinations. Methodology applies both standard-setting techniques to a 10-station OSCE completed by 102 medical students across three simulation cycles, comparing station-level and overall performance cut points. Results and conclusions analyze how failure rates change and which method better reflects OSCE difficulty and expected examiner performance.

LanguageEnglish

CategoryExam

FormatPDF

Pages7

Reading Time18 min

AccessFree to download

I. INTRODUCTION

OSCE overview and assessment reliability

Standard setting purpose and definitions

Standard setting methods and categories

ORIGINAL ARTICLE Submitted: 2 September 2023 Accepted: 9 January 2024 Published online: 2 July, TAPS 2024 , 9(3), 15-21 [https://doi.org/10.29060/TAPS.2024-9-3/OA3129](https://doi.org/10.29060/TAPS.2024-9-3/OA3129) Standard setting OSCE: A comparison of arbitrary and Hofstee methods in a low stake OSCE Uzma Khan Department of Clinical Sciences, College of Medicine, Al Rayan Colleges, Saudi Arabia Abstract Objectives: To compare the cut scores and pass/fail rates achieved by arbitrary 60% method and Hofstee method in an undergraduate year 4 end semester objective structured clinical examination (OSCE) and check the possibility of using Hofstee method of standard setting in future exams. Method: 102 medical students of year 4 underwent a 10 station OSCE exam conducted in a state of art simulation lab in 3 cycles. The cut scores were calculated using arbitrary method aiming at 60% of task achievement and by Hofstee method. The student’s obtained scores were compared for cut scores and pass rates for individual stations and the entire exam. Results: The arbitrary and Hofstee methods of standard setting leads to different cut scores. For the individual stations it was 60% vs 65-70% and for the overall score it was 60% vs 70% . The percentage of students failing the exam is 13.7% based on arbitrary scores and is 29.4% when Hofstee cut score is applied. Conclusions: The two methods lead to different cut scores and students’ failure rates. Overall, Hofstee method is more appropriate for assessing competencies in an OSCE exam in medical schools as it leads to calculation of cut scores based on the difficulty level of the station/exam and the examiners expected level of performance by the students. Keywords: Objective Structured Clinical Examination, Standard Setting, Hofstee Method, Arbitrary Method Practice Highlights 􀂃 Standard settings of OSCE identifies objective, reliable and valid cutoff scores. 􀂃 Arbitrary method scrutinises the test content and nominates the percentage of items to be answered correctly. 􀂃 Hofstee method is calculative and avoids illogical very high and low scores. 􀂃 A retrospective descriptive study design assessing applicability of Hofstee method in low stake exam in a private medical school in Saudi Arabia. 􀂃 Students’ failure rate increased with applying Hofstee standard settings in OSCE. I. INTRODUCTION Objective structured clinical examination OSCE is invented in 1975 by Harden (Harden & Gleeson, 1979) for the assessment of learners’ clinical competences and behaviors by using actors and choreographed storylines (Hodges, 2003) . He succeeded in controlling the classic variables, the patient and the examiner, that enabled him to establish a comprehensive and objective assessment (Khan et al., 2013) of competence by defining clearly what skills, attitudes, problem solving capabilities and factual knowledge are to be measured (Harden et al., 1975) . As quoted by Harden “Competency is the compound of cognitive, psychomotor and affective skills as appropriate, while competence is an attribute of aperson”(Khan et al., 2013) . During an OSCE, candidates are supposed to execute different clinical tasks in a simulated setting (Khan et al., 2013) . As a rule, the students rotate through several time limited stations in which they are expected to interact with a standardised patient (SP), mannequins or simulation models and perform a specified task meanwhile they are being assessed by examiners using standardised rating instruments (Pugh & Smee, 2013) . OSCE gets rid of many detrimental concepts in examining students, faced previously with other exam methods, by compelling them all to go through the same scope and criteria for assessment (Zayyan, 2011) . This has made it a valuable evaluative tool in medical practice, so it has been adopted in countries all over the world, in all the high stakes examinations in USA (Dwivedi et al., 2020), Canada (Pugh & Smee, 2013), and

FAQs

What is the study’s primary objective regarding OSCE standard setting?

To compare cut scores and pass/fail rates from the arbitrary 60% method versus the Hofstee method in a low-stakes OSCE, and to assess the suitability of Hofstee method for future exams.

How many students and stations were included in the method?

One hundred two year-4 medical students completed a 10-station OSCE in a simulation lab across three cycles, with cut scores computed using both standard-setting methods.

What key differences were found between arbitrary and Hofstee cut scores and pass/fail outcomes?

The two methods produced different cut scores, with higher station-level and overall thresholds under Hofstee. Student failure increased from 13.7% using arbitrary scores to 29.4% using Hofstee cut scores.

Sep 20, 2026

Uploaded by WPS_1786070896