When Doubly Robust Methods Meet Machine Learning for Estimating Average Treatment Effects for Real-World Data - A Comparative Study

Aug 5, 2026

Upload by Jiven

Observational cohort studies increasingly support comparative effectiveness research for assessing therapeutic safety, but causal inference is challenged by imbalanced covariate distributions across treatment groups. Doubly robust estimators address this by combining treatment and outcome modeling, yielding consistency when either model is correctly specified. The study evaluates popular matching, weighting, and regression variants, including machine-learning–enhanced targeted maximum likelihood estimation, through extensive simulations and a real-world application.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages29

Reading Time1 h 13 min

AccessFree to download

Introduction

Causal inference challenges in observational studies

Treatment-model and outcome-model adjustment

Doubly robust estimators and machine learning

Study design and comparison approach

Methods reviewed: matching, weighting, regression

Simulation and real-world application results

Practical guidance

arXiv :2204 . 10969v2 [ stat .ME] 26 May 2022 When Doubly Robust Methods Meet Machine Learning for Estimating Treatment E􀀋ects from Real-World Data: A Comparative Study Xiaoqing Tan∗ , Shu Yang†, Wenyu Ye‡, Douglas E. Faries‡, Ilya Lipkovich‡, and Zbigniew Kadziola‡ Abstract Observational cohort studies are increasingly being used for comparative e􀀋ectiveness research to assess the safety of therapeutics. Recently, various doubly robust methods have been proposed for average treatment e􀀋ect estimation by combining the treatment model and the outcome model via di􀀋erent vehicles, such as matching, weighting, and regression. The key advantage of doubly robust estimators is that they require either the treatment model or the outcome model to be correctly speci􀀌ed to obtain a consistent estimator of average treatment e􀀋ects, and therefore lead to a more accurate and often more precise inference. However, little work has been done to understand how doubly robust estimators di􀀋er due to their unique strategies of using the treatment and outcome models and how machine learning techniques can be combined with these estimators to boost their performance. Also, little has been understood about the challenges of covariates selection, overlapping of covariate distribution, and treatment e􀀋ect heterogeneity on the performance of these doubly robust estimators. Here we examine multiple popular doubly robust methods in the categories of matching, weighting, or regression, and compare their performance using di􀀋erent treatment and outcome modeling via extensive simulations and a real-world application. We found that incorporating machine learning with doubly robust estimators such as the targeted maximum likelihood estimator gives the best overall performance. Practical guidance on how to apply doubly robust estimators is provided. Keywords: Augmented inverse probability weighting, Double score matching, Penalized spline of propensity methods for treatment comparison, SuperLearner. ∗ Department of Biostatistics, University of Pittsburgh, Pittsburgh, PA, USA †Department of Statistics, North Carolina State University, Raleigh, North Carolina, USA ‡Eli Lilly and Company, Indianapolis, Indiana, USA 1 Introduction Randomized control trials (RCTs) are considered to be the gold standard for establishing the causal e􀀋ects of interventions. They evaluate interventions among comparable groups. However, sometime it would be impossible to conduct RCTs due to limited resources or ethical issues. Observational studies, on the other hand, examine e􀀋ects in \real world"settings without manipulation. As there is no intervention, some individuals with certain characteristics may have a di􀀋erent probability of being exposed to treatment than others, meaning that the covariate information between treatment groups may be highly imbalanced. Therefore, it's important to adjust for covariate imbalance issues in observational studies. There are two ways of adjustment for observational studies. The 􀀌rst kind is based on the treatment model, also known as the propensity score (PS) model where the PS is de􀀌ned to be the probability of being treated given covariates. The common inverse propensity treatment weighted estimator falls into this category. The idea of weighting is to create a weighted pseudo-population where treatments are \randomized". Another kind is based on the outcome model. This outcome imputation approach tries to impute the missing potential outcomes based on outcome modeling. Estimators based on PS modeling require the correct treatment model, and estimators based on outcome modeling require the correct outcome model. In practice, it's common to use linear models for PS and outcome modeling. This, however, could be problematic because linearity may be an inappropriate assumption for PS and outcomes when the response surface is nonlinear. To account for potential nonlinearity, more 􀀍exible models are needed to be considered. Doubly r

FAQs

Why are randomized control trials difficult, and what problem does observational data create?

Randomized control trials may be infeasible due to limited resources or ethical issues. Observational studies lack intervention, so treated and untreated groups can have highly imbalanced covariates, requiring careful adjustment.

What makes doubly robust estimators attractive for estimating average treatment effects?

They can produce consistent estimates of average treatment effects if either the treatment model or the outcome model is correctly specified. This enables more accurate and often more precise inference.

How does combining machine learning with doubly robust estimators affect performance?

In the comparative study, incorporating machine learning with doubly robust estimators—especially targeted maximum likelihood estimation—achieves the best overall performance for estimating average treatment effects.

Aug 5, 2026

Upload by Jiven