Purpose and Value of Subject Matter Experts - Background - Selecting SMEs - Job Analysis - Exam Development, Review and Validation - Development of Assessment Center Exercises

Sep 18, 2026

Uploaded by Mimi

Purpose and Value of Subject Matter Experts explains how subject-matter experts (SMEs) contribute to creating and validating selection tools. It details SME roles across selection and promotion, emphasizing their deep knowledge of job duties, responsibilities, and required knowledge, skills, and abilities. The document covers selecting appropriate SME types (supervisors or incumbents), conducting job analysis through interviews, observations, and JAQs, and using SME input to develop valid exams. It further describes review procedures and validation using CVRs and Angoff ratings, and supports development of assessment center exercises with job-relevant scenarios.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages3

Reading Time8 min

AccessFree to download

Background

Selecting SMEs

SMEs and the Development and Validation of Selection Tools

Job Analysis

Exam Development, Review and Validation

Development of Assessment Center Exercises

Purpose and Value of SubjectMatter Experts Background Subject-matter experts (SMEs) are often utilized to assist in the development and validation of selection tools. In the context of selection and promotion, SMEs provide invaluable insight on the job or rank in question. SMEs have an indepth understanding of a job or rank and are able to thoroughly and accurately discuss duties and responsibilities of a job, knowledge requirements, skill requirements, ability requirements, and other pertinent information. With this expertise, SMEs are able to assist with various phases of the development and validation of tests and assessments. Selecting SMEs The selection of the type of SMEs may vary depending on the purpose of their involvement. SMEs may be incumbents or supervisors. In situations where test content or assessment content is being developed or reviewed, it is most appropriate to use SMEs who are supervisors. In situations where the job is being discussed for job analysis purposes, it is appropriate to use incumbent SMEs along with supervisor SMEs. S MEs and the Development and Validation of Selection Tools SMEs are used for various purposes during several phases of the development and validation of selection tools. The fundamental building block in starting the development of any selection tool is a thorough job analysis. According to the Equal Employment Opportunity Commission’s (EEOC) “Uniform Guidelines on Employee Selection Procedures,”in order for selection tools to be valid, they must measure knowledge, skills and abilities that area “necessary prerequisite to performance of critical or important work behavior(s) .” In order to ascertain this information, a job analysis must be conducted. Job Analysis During the job analysis phase, SMEs have an integral role in providing information about the job via interviews, on-the-job observations, and job analysis questionnaires (JAQs) . SMEs are interviewed to gather information on the tasks and responsibilities required for the job, critical knowledge, skill, and ability requirements, along with any other critical incident information that is relevant. Also, SMEs may be observed while they are performing their job to obtain firsthand knowledge and information about the job being analyzed. Utilizing the information gathered from the interviews and observations, a JAQ may be developed to distribute to incumbents, supervisors and any other relevant SME group. The JAQ would require a representative sample of SMEs to rate the importance and frequency of tasks and the importance of various knowledge, skill and ability areas required for successful performance in the job. It is evident that the role of SMEs in the job analysis phase is crucial, and their participation allows for a thorough and accurate understanding of the job. Using the information gathered from the interviews, observations and JAQs, valid selection tools can be developed. Exam Development, Review and Validation After data from the job analysis have been analyzed, the essential job knowledge areas of the position are known. In order to develop a custom job knowledge exam, sources (e.g., textbooks, internal policy manuals, state laws, local ordinances, etc. ) must be selected that contain the essential job knowledge areas of the position. SMEs are used to help in selecting agency-relevant sources that include the essential job knowledge areas for the position. For example, if it is essential for the position tobe knowledgeable on state laws, SMEs would assist in selecting the specific state laws that are most relevant for inclusion in the exam. The sources that SMEs assist in selecting would then be communicated to the candidates. The candidates prepare for the exam by studying the sources on this list. The exam items are developed directly from the sources that were selected and communicated to the candidates. Another phase in which the use of SMEs is important is in the review of various s

FAQs

What roles do SMEs play in developing and validating selection tools?

SMEs provide expert insight into job duties and requirements, supporting development and validation across multiple phases, including job analysis and exam construction.

How are SMEs selected for different selection tool activities?

The involvement type depends on the goal: supervisors are most appropriate when developing or reviewing test content, while incumbent SMEs may be used alongside supervisors for job analysis.

How does SME input support exam validation and scoring?

SMEs assign content-validity ratios (CVRs) to judge essentiality and provide Angoff ratings to estimate the performance of minimally qualified incumbents, which informs the exam cut score.

Sep 18, 2026

Uploaded by Mimi