EE460 - Tornado Diagrams Only Tell Us 5% of the Story - Recommendations for More Informative Reporting of One-Way Sensitivity Analysis

Oct 26, 2026

Upload by Aurelia

Deterministic sensitivity analysis is widely used in health technology appraisals to test model robustness by varying individual parameters within specified ranges. This study examines recent NICE health technology appraisals to assess how the parameter ranges were justified and whether tornado diagrams report results only at extreme values or also at intermediate points. Across 26 usable appraisals, tornado diagrams display key outputs solely at extremes, motivating stepwise approaches to provide more granular insight into parameter–outcome dynamics.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages1

Reading Time3 min

AccessFree to download

Introduction

Objectives

Methods

Results

Data used to inform ranges

Information presented by the Tornado diagram

Extra value is hidden when using classical DSA methods

Recommendations

Conclusions

EE460 Tornado Diagrams Only Tell Us 5% of the Story: Recommendations for More Informative Reporting of One-Way Sensitivity Analysis Taylor M 1 , Davis R 1 1 York Health Economics Consortium, York, United Kingdom INTRODUCTION ▪ Deterministic sensitivity analysis (DSA) is commonly employed within health technology appraisals (HTA) to assess the robustness of the model outcome to changes in individual key parameters of the model. Within such an exercise, parameters are varied within a specified range sequentially whilst holding other parameter estimates constant. ▪ A common output of DSA is a tornado diagram. Classically, the Tornado diagram displays the change in the model results when varying the parameter estimate between a lower and upper extreme value. These extreme values which determine the range should ideally be based on observed uncertainty for the parameter input (e.g. confidence intervals (CI) or standard errors (SE)) rather than arbitrary values. Objectives: ▪ By examining recent HTAs which conducted DSA, this study investigates what information the range was based on, and whether the Tornado diagram presented displays the model results solely when utilising extreme parameter values or for other values. This study explores the usefulness of the Tornado diagram produced using classical DSA methods, and whether alternative approaches should be recommended. METHODS ▪ The National Institute for Health and Care Excellence (NICE) website was searched to source HTAs in the last three years that mentioned either ‘deterministic sensitivity analysis’ or ‘one-way sensitivity analysis’. For each HTA highlighted via the search methods, two key pieces of information was extracted: 1. The data that had been used to inform the range of parameters selected for DSA. 2. Whether the tornado diagram presented the key output (e.g. incremental costeffectiveness ratio (ICER) or net health benefit (NHB)) solely for the extreme values in the range, or whether information was provided for the intermediate values in the range. RESULTS ▪ 26 HTAs were identified which mentioned at least one of the search terms. ▪ 2 HTAs were removed from the extraction results due to high levels of redaction which rendered them unusable. Data used to inform ranges ▪ 25% of HTAs used only 95%CIs or SEs to inform the upper and lower range estimates. ▪ 54% of HTAs used e95%CIs, SEs or arbitrary values to inform the upper and lower range estimates. ▪ 17% of HTAs used only arbitrary values to inform the upper and lower range estimates. Information presented by the Tornado diagram ▪ In all usable cases the Tornado diagram presented the key output e.g. ICER or NHB for the extreme values in the range. In no instance was results provided for the intermediate values. Extra value is hidden when using classical DSA methods ▪ The results indicate that the classic tornado diagram is the preferred visual aid to represent one-way sensitivity analysis. ▪ The corollary being that the value of leveraging more advanced techniques is being unrealised , value such as understanding how model outcomes change when utilising parameter estimates that are in the range between extremes. ▪ Employing such an approach elucidates a more nuanced relationship between a parameter and the model outcome with respect to linearity. ▪ When utilsing such an approach, the user can conclude at which point along the parameter range that the intervention begins to dominate the comparator. ▪ A benefit of obtaining such information is that more precise conclusions can be drawn from DSA and superficial ones discarded. RECOMMENDATIONS ▪ The classic DSA approach assesses the change in the outcome of interest when utilising parameter estimates that have a 2.5% chance of occurring (i.e. at each end of the range) . When these results are presented in a tornado diagram, they take the form presented in Figure 1. ▪ A recommendation to advance on the classic DS

FAQs

What does deterministic sensitivity analysis typically do in health technology appraisals?

It varies one key model parameter sequentially within a specified range while holding other parameter estimates constant, to assess robustness of the model outcome.

How are tornado diagrams used in the reviewed health technology appraisals?

In all usable cases, the tornado diagram presents the key output only for the extreme values in the range, without providing results for intermediate values.

What recommendation is made to improve tornado diagram reporting?

Advance beyond classical one-way deterministic sensitivity analysis by using stepwise DSA, which evaluates intermediate uniform steps along the range to show a more granular parameter–outcome relationship.

Oct 26, 2026

Upload by Aurelia