4 rd Regional Meeting for WOAH Reference Centres in Asia and the Pacific - Diagnostic test validation: Lessons learned

Oct 2, 2026

Regional meeting experience sharing focused on diagnostic test validation and how to judge an assay’s fitness for a stated purpose. The session outlined the validation pathway from defining purpose through optimization, standardization, repeatability, analytical and diagnostic sensitivity/specificity, cut-offs, and reproducibility. It also addressed key parameters and practical challenges such as selecting case definitions, handling assay evidence gaps, variability within and between assays, control limits, reference test imperfections, latent class approaches, and ensuring source and target populations are comparable.

LanguageEnglish

CategoryExam

FormatPDF

Pages7

Reading Time18 min

AccessFree to download

Experience sharing: Diagnostic test validation - Lessons learned

WOAH validation pathway

Parameters and challenges (purpose, analytical performance, cut-off, reproducibility)

Scope, limitations, and study design considerations

4 rd Regional Meeting for WOAH Reference Centres in Asia and the Pacific, Japan, 16-19 July 2024 Experience sharing: Diagnostic test validation: Lessons learned Axel Colling et al., Australian Centre for Disease Preparedness (former AAHL), CSIRO WOAH Collaborating Centre for Diagnostic Test Validation Science Asia Pacific Region Test validation is a process that determines the fitness of an assay or test, which has been properly developed, optimised and standardised, for an intended purpose.”(WOAH 1.1.6., 2024) WOAH validation pathway 1) Definition of purpose 2) Optimization 3) Standardization 4) Repeatability 5) Analytical Sensitivity 6) Analytical Specificity 7) Cut-off(s) 8) Diagnostic Sensitivity 9) Diagnostic Specificity 10) Reproducibility 11) Fitness for purpose (Chapter 1.1.6.) Parameter • Test purpose, e.g. screening assay (high Se) or confirmatory assay (high Sp) • Case definition (consistency) • ASp (selectivity, exclusivity, inclusivity) • ASe (Limit of detection) Challenge • Not defined • Too many purposes • Evidence to support purpose is sometimes missing or incomplete • Define species and specimen (domestic chicken, tracheal/cloacal swab • What constitutes an infected and non-infected animal? • Which test(s) are chosen to define status, clinical signs, etc.? • ASp is the ability of the assay to distinguish the target analyte (e.g. antibody, organism or genomic sequence) from non-target analytes, including matrix components • Estimated minimum amount of analyte in a specified matrix that would produce a positive result within at least a specified percent of the time • In serology ASe is often given as a relative value to a reference test • For molecular tests it is important to include the extraction procedure in the evaluation Parameter • Repeatability (variation within the laboratory) • Diagnostic sensitivity (Percentage of known infected animals that test positive in the assay) • Diagnostic specificity (Percentage of known uninfected animals that test negative in the assay) Challenge • Intra-and inter-assay variation in a laboratory (C-, C+, C++) • Coefficient of variation • Upper and lower control limits established? • Established procedure when values fall outside limits? • How many samples need to be tested for robust DSe and DSp estimates? (Chapter 1.1.6.) • Experimental infection • No repeated sampling from same animal • Vaccinated animals • Results can over-or underestimate true DSe of assay • Use of negative samples from different negative source population (can result in over-or underestimation of DSp in target population) Parameter • Cut-off • Reference test imperfect • Reproducibility • Define scope and limitations Challenge • Single or multiple cut-offs (positive, negative, intermediate) • Overlapping results from positive and negative populations? • Mean of negative population +2/3StD • ROC curve (samples from infected and non-infected animals) • Impact of cut-off on DSe and DSp • If no perfect reference test is available a Latent Class Analysis is needed • >3 laboratories (interlaboratory variation) • Panel of well-characterised positive and negative samples • What can be expected from a diagnostic test (scope) and what cannot be expected (limitations)? • Is source population (where test was validated) similar to target population (where test is applied)? Thanks for your attention

FAQs

What does diagnostic test validation aim to establish?

It determines the fitness of an assay or test—properly developed, optimized, and standardized—for its intended purpose.

What are the main steps in the WOAH validation pathway?

The pathway covers purpose definition, optimization, standardization, repeatability, analytical sensitivity and specificity, cut-off(s), diagnostic sensitivity and specificity, reproducibility, and overall fitness for purpose.

Why can estimating diagnostic sensitivity and specificity be challenging?

Challenges include intra- and inter-assay variation, limited or missing evidence to support the purpose, imperfect reference tests, overlapping results between populations, and issues with sample selection and control limits. Results may over- or underestimate true performance if study design is not appropriate.

Oct 2, 2026

Uploaded by cryingin4k