Situational Judgment Tests: Constructs Assessed and a Meta-Analysis of Their Criterion-Related Validities

Sep 18, 2026

Uploaded by Mary Man

Situational judgment tests (SJTs) are used to select employees, yet prior research often reports only overall scores and provides limited detail about the specific constructs assessed. This work promotes a construct-level focus by identifying and classifying construct domains measured by SJTs into a content-based typology. A meta-analysis then estimates criterion-related validity for each domain and tests moderators, showing SJTs frequently assess leadership and interpersonal skills. Evidence also suggests that construct-criterion matching improves validity and that video-based SJTs may yield stronger validity than pencil-and-paper formats.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages35

Reading Time1 h 28 min

AccessFree to download

Introduction

Purpose and construct-level focus

SJT measurement approach and formats

Construct Typology and Classification

Content-based typology of assessed constructs

Meta-Analysis of Validities

Criterion-related validity by construct domain

Moderators and validity improvements

PERSONNEL PSYCHOLOGY 2010, 63, 83–117 SITUATIONAL JUDGMENT TESTS: CONSTRUCTS ASSESSED AND A META-ANALYSIS OF THEIR CRITERION-RELATED VALIDITIES MICHAEL S. CHRISTIAN Eller College of Management University of Arizona BRYAN D. EDWARDS William S. Spears School of Business Oklahoma State University JILL C. BRADLEY Craig School of Business California State University, Fresno Situational judgment tests (SJTs) are a measurement method that maybe designed to assess a variety of constructs. Nevertheless, many studies fail to report the constructs measured by the situational judgment tests in the extant literature. Consequently, a construct-level focus in the situational judgment test literature is lacking, and researchers and practitioners know little about the speciﬁc constructs typically measured. Our objective was to extend the efforts of previous researchers (e.g., McDaniel, Hartman, Whetzel, & Grubb, 2007; McDaniel & Ngyuen, 2001; Schmitt & Chan, 2006) by highlighting the need for a construct focus in situational judgment test research. We identiﬁed and classiﬁed the construct domains assessed by situational judgment tests in the literature into a content-based typology. We then conducted a meta-analysis to determine the criterion-related validity of each construct domain and to test for moderators. We found that situational judgment tests most often assess leadership and interpersonal skills and those situational judgment tests measuring teamwork skills and leadership have relatively high validities for overall job performance. Although based on a small number of studies, we found evidence that (a) matching the predictor constructs with criterion facets improved criterion-related validity; and (b) videobased situational judgment tests tended to have stronger criterion-related Authors’ Note. This paper is based in part on the master’s thesis of Michael S. Christian, which was chaired by Bryan D. Edwards. An earlier version of this paper was presented atthe 2007 Annual Conference of the Society for Industrial and Organizational Psychology. We are grateful to Winfred Arthur, Jr., Ronald Landis, Michael Burke, and Filip Lievens for reviewing previous drafts of this article and providing valuable suggestions. We also thank Michael McDaniel, Phillip Bobko, and Edgar Kausel for their helpful comments and suggestions on this project. Finally, we acknowledge the work of Jessica Siegel, Helen Terry, and Adela Garza. Correspondence and requests for reprints should be addressed to Michael S. Christian Eller College of Management, University of Arizona, Department of Management and Organizations, McClelland Hall, PO Box 210108, Tucson, AZ 85721-0108; [msc@email.arizona.edu](msc@email.arizona.edu). 2010 Wiley Periodicals, Inc. 84 PERSONNEL PSYCHOLOGY validity than pencil-and-paper situational judgment tests, holding constructs constant. Implications for practice and research are discussed. Situational judgment tests (SJTs) have a long history of use for employee selection (e.g., File, 1945; File & Remmers, 1971; Motowidlo, Dunnette, & Carter, 1990; Weekley & Jones, 1997, 1999) . An SJT is a measurement method typically composed of job-related situations or scenarios that describe a dilemma or problem requiring the application of relevant knowledge, skills, abilities, or other characteristics (KSAOs) to solve. SJT items may be presented in written, verbal, video-based, or computer-based formats (e.g., Clevenger, Pereira, Wiechmann, Schmitt,& Schmidt-Harvey, 2001; Motowidlo et al., 1990), and usually contain options representing alternative courses of action from which the test taker chooses the most appropriate response. Despite the widespread use of SJTs, even a cursory review of the literature reveals that test developers and researchers often give little attention to the constructs measured by SJTs and instead tend to report results based on overall (or composite) SJT scores. Nevertheless, because SJTs are

FAQs

Why is a construct-level focus important in situational judgment test research?

Because SJTs are measurement methods, understanding the specific constructs they assess is essential for interpreting psychometric properties such as reliability, validity, and subgroup differences.

How does the study determine which constructs SJTs assess?

It identifies and classifies construct domains assessed by SJTs in the literature into a content-based typology.

What does the meta-analysis show about criterion-related validity?

SJTs most often assess leadership and interpersonal skills, and tests measuring teamwork skills and leadership show relatively high validities for overall job performance.

Sep 18, 2026

Uploaded by Mary Man