Double/Debiased Machine Learning for Treatment and Structural Parameters

Aug 5, 2026

Upload by Ophelia

Revisiting the classic semiparametric inference problem for a low-dimensional target parameter θ0 under high-dimensional nuisance functions η0, this work studies settings where traditional complexity conditions fail. Naively plugging machine-learning estimates of η0 into estimating equations yields heavy bias, breaking N−1/2 consistency. The paper shows this bias can be eliminated using Neyman-orthogonal moments/scores and cross-fitting. The resulting double/debiased ML methods produce asymptotically normal, approximately unbiased estimators, enabling valid confidence statements and applications to multiple causal and treatment-effect models.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages71

Reading Time2 h 59 min

AccessFree to download

Introduction and Motivation

Motivation

Example: Partially Linear Regression

Summary of Core Ideas and Contributions

arXiv : 1608 .00060v7 [ stat .ML] 3 Nov 2024 Double/Debiased Machine Learning for Treatment and Structural Parameters Victor Chernozhukov†, Denis Chetverikov‡, Mert Demirer† Esther Duflo†, Christian Hansen§ , Whitney Newey†, James Robins⋆ † Massachusetts Institute of Technology, 50 Memorial Drive, Cambridge, MA, 02139, USA E-mail: [vchern@mit.edu](vchern@mit.edu) , [mdemirer@mit.edu](mdemirer@mit.edu) , [duflo@mit.edu](duflo@mit.edu) , [wnewey@mit.edu](wnewey@mit.edu) † University of California Los Angeles, 315 Portola Plaza, Los Angeles, CA 90095 E-mail: [chetverikov@econ.ucla.edu](chetverikov@econ.ucla.edu) § University of Chicago, 5807 S. Woodlawn Ave. , Chicago, IL 60637 E-mail: [chansen1@chicagobooth.edu](chansen1@chicagobooth.edu) ⋆ Harvard University, 677 Huntington Avenue Boston, Massachusetts 02115 E-mail: [robins@hsph.harvard.edu](robins@hsph.harvard.edu) Received: June 2014 Summary We revisit the classic semiparametric problem of inference on a low dimensional parameter θ0 in the presence of high-dimensional nuisance parameters η0 . We depart from the classical setting by allowing for η0 to be so high-dimensional that the traditional assumptions, such as Donsker properties, that limit complexity of the parameter space for this object break down. To estimate η0 , we consider the use of statistical or machine learning (ML) methods which are particularly well-suited to estimation in modern, very high-dimensional cases. ML methods perform well by employing regularization to reduce variance and trading off regularization bias with overfitting in practice. However, both regularization bias and overfitting in estimating η0 cause a heavy bias in estimators of θ0 that are obtained by naively plugging ML estimators of η0 into estimating equations for θ0 . This bias results in the naive estimator failing to be N −1/2 consistent, where N is the sample size. We show that the impact of regularization bias and overfitting on estimation of the parameter of interest θ0 can be removed by using two simple, yet critical, ingredients: (1) using Neyman-orthogonal moments/scores that have reduced sensitivity with respect to nuisance parameters to estimate θ0 , and (2) making use of cross-fitting which provides an efficient form of data-splitting. We call the resulting set of methods double or debiased ML (DML) . We verify that DML delivers point estimators that concentrate in a N −1/2-neighborhood of the true parameter values and are approximately unbiased and normally distributed, which allows construction of valid confidence statements. The generic statistical theory of DML is elementary and simultaneously relies on only weak theoretical requirements which will admit the use of a broad array of modern ML methods for estimating the nuisance parameters such as random forests, lasso, ridge, deep neural nets, boosted trees, and various hybrids and ensembles of these methods. We illustrate the general theory by applying it to provide theoretical properties of DML applied to learn the main regression parameter in a partially linear regression model, DML applied to learn the coefficient on an endogenous variable in a partially linear instrumental variables model, DML applied to learn the average treatment effect and the average treatment effect on the treated under unconfoundedness, and DML applied to learn the local average treatment effect in an instrumental variables setting. In addition to these theoretical applications, we also illustrate the use of DML in three empirical examples. 2 CCDDHNR 1. INTRODUCTION AND MOTIVATION 1.1. Motivation We develop a series of simple results for obtaining root-N consistent estimation, where Nis the sample size, and valid inferential statements about a low-dimensional parameter of interest, θ0 , in the presence of a high-dimensional or “highly complex” nuisance parameter, η0 . The parameter of interest will typically be a causal parameter or treatment effect parameter, a

FAQs

Why does naive plugging of ML nuisance estimates fail for estimating θ0?

Regularization bias and overfitting in estimating η0 introduce substantial bias into estimators obtained by naively inserting ML nuisance estimates into the estimating equations for θ0.

What two ingredients make double/debiased ML remove the heavy bias?

It uses Neyman-orthogonal moments/scores with reduced sensitivity to nuisance estimation and cross-fitting to implement an efficient form of data splitting.

What kinds of results does the method enable for the estimator of θ0?

It yields point estimators that concentrate in an N−1/2 neighborhood of the truth, are approximately unbiased, and are asymptotically normal, allowing construction of valid confidence statements.

Aug 5, 2026

Upload by Ophelia