[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-124074-en":3,"doc-seo-124074-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},124074,687197207639,"Asher","https://ap-avatar.wpscdn.com/davatar_a8503ba1806abce46bf441b54a3ca4cd",8,"Research & Report","Neyman Meets Causal Machine Learning - Experimental Evaluation of Individualized Treatment Rules","A paper revisits Neyman’s century-old randomized experimental framework to evaluate the causal impact of individualized treatment rules (ITRs) obtained from modern causal machine learning. It addresses additional uncertainty introduced by cross-fitting in the training process and shows that Neyman’s approach can be used for any ITR regardless of how it was learned. The work also compares ex-post versus ex-ante experimental evaluation, finding cases where ex-post testing is more efficient, and reinforcing the continued relevance of repeated sampling for contemporary causal inference.","arXiv :2404 . 17019v1 [ stat .ME] 25 Apr 2024  \nResearch Article Open Access  \nMichael Lingzhi Li* and Kosuke Imai  \nNeyman Meets Causal Machine Learning: Experimental Evaluation of Individualized Treatment Rules  \nDOI: DOI, Received .. ; revised .. ; accepted ..  \nAbstract: A century ago, Neyman showed how to evaluate the efficacy of treatment using a randomized experiment under a minimal set of assumptions. This classical repeated sampling framework serves as a basis of routine experimental analyses conducted by today’s scientists across disciplines. In this paper, we demonstrate that Neyman’s methodology can also be used to experimentally evaluate the efficacy of individualized treatment rules (ITRs), which are derived by modern causal machine learning algorithms. In particular, we show how to account for additional uncertainty resulting from a training process based on cross-fitting. The primary advantage of Neyman’s approach is that it can be applied to any ITR regardless of the properties of machine learning algorithms that are used to derive the ITR. We also show, somewhat surprisingly, that for certain metrics, it is more efficient to conduct this ex-post experimental evaluation of an ITR than to conduct an ex-ante experimental evaluation that randomly assigns some units to the ITR. Our analysis demonstrates that Neyman’s repeated sampling framework is as relevant for causal inference today as it has been since its inception.  \nKeywords: causal inference, machine learning, individualized treatment rule, policy evaluation, repeated sampling  \nMSC: 62G05  \n1 Introduction  \nNeyman’s seminal 1923 paper introduced two foundational ideas in causal inference [1] . First, Neyman developed a formal notation for potential outcomes and defined the average treatment effect (ATE) as a causal quantity of interest. Second, he showed how randomization of treatment assignment alone can be used to establish the unbiasedness and estimation uncertainty of the standard difference-in-means estimator. Since then, combined with the additional assumption of random sampling of units, Neyman’s repeated sampling framework has served as a basis of routine experimental analyses conducted by scientists across many disciplines.  \nOver the past two decades, however, the causal inference literature has gone beyond the ATE. Specifically, the realization that the same treatment can have varying impacts on different individuals led to the development of statistical methods and machine learning algorithms for estimating heterogeneous treatment effects [e.g., 2–5] . Furthermore, a number of researchers have developed various methods for deriving data-driven individualized treatment rules (ITRs) [e.g. 6–13] . With an increasing availability of granular data and modern computing power, these ITRs are becoming popular in business, medicine, politics, and even public policy.  \nIn this paper, we demonstrate that Neyman’s repeated sampling framework is still relevant for today’s causal machine learning methods. We show how the framework can be used to experimentally evaluate the  \n*Corresponding Author: Michael Lingzhi Li: Harvard Business School; E-mail: [mili@hbs.edu](mili@hbs.edu)[ ](mili@hbs.edu)Kosuke Imai: Harvard University; E-mail: [imai@harvard.edu](imai@harvard.edu)  \n2 ~~ ~~ Michael Lingzhi Li and Kosuke Imai, Neyman Meets Causal Machine Learning   \nefficacy of any ITRs (including those obtained with machine learning algorithms via cross-fitting) under a minimal set of assumptions. While some of our formal results are originally derived in our previously published work [14] or follow directly from them, we focus on the intuition behind those theoretical results to facilitate the future extensions to other settings.  \nWe also show, using Neyman’s framework, that it is not always statistically more efficient to evaluate an ITR by conducting a new randomized experiment where the treatment is the administration of the ITR itself (i.e. , ex-ante evaluation)","cbCaijA3TztW39oi","https://ap.wps.com/l/cbCaijA3TztW39oi","pdf",574341,1,21,"English","en",105,"# Introduction\n# Neyman’s Repeated Sampling Framework","[{\"question\":\"How does Neyman’s original framework support causal evaluation in experiments?\",\"answer\":\"It uses randomization of treatment assignment to establish unbiasedness and quantify estimation uncertainty for standard estimators, built on potential outcomes and repeated sampling assumptions.\"},{\"question\":\"How are individualized treatment rules (ITRs) evaluated in this work?\",\"answer\":\"The paper shows how to experimentally assess the efficacy of any ITR using Neyman’s repeated sampling framework, including ITRs derived via machine learning with cross-fitting.\"},{\"question\":\"Why can ex-post evaluation outperform ex-ante evaluation for some metrics?\",\"answer\":\"For certain performance metrics, using data from an existing randomized controlled trial to conduct ex-post evaluation is more efficient than running a new experiment that randomly assigns the ITR ex-ante.\"}]","Neyman Meets Causal Machine Learning - Experimental Evaluation of Individualized Treatment Rules | PDF",1785820200,53,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"neyman-meets-causal-machine-learning-experimental-evaluation-of-individualized-treatment-rules","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/neyman-meets-causal-machine-learning-experimental-evaluation-of-individualized-treatment-rules/124074/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-04",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"How does Neyman’s original framework support causal evaluation in experiments?","Question",{"text":75,"@type":76},"It uses randomization of treatment assignment to establish unbiasedness and quantify estimation uncertainty for standard estimators, built on potential outcomes and repeated sampling assumptions.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How are individualized treatment rules (ITRs) evaluated in this work?",{"text":80,"@type":76},"The paper shows how to experimentally assess the efficacy of any ITR using Neyman’s repeated sampling framework, including ITRs derived via machine learning with cross-fitting.",{"name":82,"@type":73,"acceptedAnswer":83},"Why can ex-post evaluation outperform ex-ante evaluation for some metrics?",{"text":84,"@type":76},"For certain performance metrics, using data from an existing randomized controlled trial to conduct ex-post evaluation is more efficient than running a new experiment that randomly assigns the ITR ex-ante.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]