Introducing a Smartphone Tele-Objective Structured Clinical Examination to Support High-Stakes Competency Decisions - A Quasi-Experimental Study and Curricular Implications

Q: What problem does the study address about high-stakes OSCEs?

High-stakes OSCEs are resource-intensive and may constrain access and equity, particularly for candidates from remote locations.

Q: How was the smartphone tele-OSCE evaluated?

The study used a quasi-experimental historical-control design comparing a 2021 tele-OSCE with a 2019 in-person OSCE, measuring overall scores, pass/fail decisions, acceptability, costs, total time, and logistics, along with psychometric checks.

Q: What were the key findings regarding assessment performance and logistics?

Tele-OSCE mean scores were lower than the in-person OSCE, while pass rates remained similarly high; interexaminer agreement was moderate to high. Tele-OSCE reduced the need for candidate travel, lowered on-site density, and improved implementation feasibility and acceptability.

Sep 29, 2026

Uploaded by A glass of water

High-stakes Objective Structured Clinical Examinations (OSCEs) are resource-intensive and can limit access and equity, especially for remote candidates. A smartphone-based tele-OSCE may reduce logistical burdens while preserving assessment quality. This quasi-experimental historical-control study compared a 2021 tele-OSCE with a 2019 in-person OSCE for 5th-year medical students, assessing scores and pass/fail decisions, acceptability, direct costs, examination time, and logistics, including psychometric agreement.

LanguageEnglish

CategoryHealthcare

FormatPDF

Pages14

Reading Time35 min

AccessFree to download

Background

Objective

Methods

Study design

Outcomes and psychometric analyses

Results

Assessment outcomes and pass/fail

Costs, time, and logistics

Acceptability

Conclusions

Implementation implications

Introducing a Smartphone Tele-Objective Structured Clinical Examination to Support High-Stakes Competency Decisions: A Quasi-Experimental Study and Curricular Implications Journal of Medical Education and Curricular Development Volume 13: 1-14 © The Author(s) 2026 Article reuse guidelines: [sagepub.com/journals-permissions](sagepub.com/journals-permissions)[ ](sagepub.com/journals-permissions)[DOI: 10.1177/23821205251413401](DOI: 10.1177/23821205251413401)[ ](DOI: 10.1177/23821205251413401)[journals.sagepub.com/home/mde](journals.sagepub.com/home/mde) Xiaozhi Wang 1,*, Junjie Du2,* , Binlin Luo3, Liling Chen4, Huanhuan Chen5, Surong Jiang6, Wei Sun7, Lei Zhou 1, Lars Konge8,9, Hua Huang4 and Qiang Ding4 Abstract Background: High-stakes Objective Structured Clinical Examinations (OSCEs) are resource-intensive and may constrain access and equity, especially for candidates from remote locations. Smartphone-based tele-OSCEs could reduce logistical burdens while maintaining assessment quality. Objective: To evaluate whether a smartphone tele-OSCE yields assessment outcomes comparable to an in-person OSCE while improving implementation efﬁciency, costs, and acceptability within an undergraduate medical curriculum. Methods: We conducted a quasi-experimental historical-control study (2021 tele-OSCE vs 2019 in-person OSCE) in 5thyear medical students at a single university in China. The tele-OSCE comprised 2 stations (history taking, clinical reasoning) aligned with the course blueprint. Primary outcomes included overall scores and pass/fail decisions; secondary outcomes included examiner/standardized patient (SP)/student acceptability, direct per-candidate costs, total examination time, and logistical metrics. Psychometric analyses included interexaminer correlations and descriptive consistency checks. Results: Of 176 candidates scheduled for the 2021 tele-OSCE, 164 without prior online-OSCE exposure were analyzed; 272 in-person candidates from 2019 served as historical controls. Students in the tele-OSCE cohort obtained lower mean scores than those in the 2019 in-person OSCE cohort (65.6 ± 11.2 vs 72.0 ± 10.6; P < .001; Cohen’s d = 0.59), although pass rates remained similarly high in both years. Interexaminer correlations across 4 cases ranged from r = .47 to .93 (all P < .05), indicating moderate-to-high agreement. Direct costs per candidate were modestly higher for tele-OSCE than in-person (USD45.22 vs 40.13; + 12.7%). The tele-OSCE eliminated the need for candidates to travel (commuting) and reduced on-site density; most students, SPs, and examiners reported positive acceptability. Conclusions: A smartphone tele-OSCE can support curriculum-integrated, high-stakes competency decisions with performance outcomes that remain within an acceptable range relative to the conventional format, while improving logistical feasibility. We provide implementation details and practical guidance to facilitate replication in similar curricular settings. Keywords tele-OSCE, smartphone-based assessment, high-stakes examinations, objective structured clinical examination, curricular implementation Received: 30 September 2025; accepted: 16 December 2025 1 Department of Cardiology, The First Afﬁliated Hospital with Nanjing Medical University, Nanjing, China 2Department of Cardiovascular Surgery, The First Afﬁliated Hospital with Nanjing Medical University, Nanjing, China 3Department of Plastic and Burn Surgery, The First Afﬁliated Hospital with Nanjing Medical University, Nanjing, China 4The First School of Clinical Medicine, Nanjing Medical University, Nanjing, China 5Department of Endocrinology, The First Afﬁliated Hospital with Nanjing Medical University, Nanjing, China 6Department of Geriatric Cardiology, The First Afﬁliated Hospital with Nanjing Medical University, Nanjing, China 7Department of Gynecology, The First Afﬁliated Hospital with Nanjing Medical University, Nanjing, China 8Guangdong Academy for

FAQs

What problem does the study address about high-stakes OSCEs?

High-stakes OSCEs are resource-intensive and may constrain access and equity, particularly for candidates from remote locations.

How was the smartphone tele-OSCE evaluated?

The study used a quasi-experimental historical-control design comparing a 2021 tele-OSCE with a 2019 in-person OSCE, measuring overall scores, pass/fail decisions, acceptability, costs, total time, and logistics, along with psychometric checks.

What were the key findings regarding assessment performance and logistics?

Tele-OSCE mean scores were lower than the in-person OSCE, while pass rates remained similarly high; interexaminer agreement was moderate to high. Tele-OSCE reduced the need for candidate travel, lowered on-site density, and improved implementation feasibility and acceptability.

Sep 29, 2026

Uploaded by A glass of water