[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-seo-201891-105":3,"detail-sidebar-cat-0-en-105":81,"doc-detail-201891-en":131},{"code":4,"msg":5,"data":6},0,"ok",{"site_id":7,"language":8,"slug":9,"title":10,"keywords":11,"description":12,"schema_data":13,"social_meta":74,"head_meta":76,"extra_data":78,"updated_unix":80},105,"en","can-variation-in-subgroups-average-treatment-effects-explain-treatment-effect-heterogeneity-evidence-from-a-social-experiment-draft-paper-with-appendix-figures","Can Variation in Subgroups’ Average Treatment Effects Explain Treatment Effect Heterogeneity? Evidence from a Social Experiment - Draft paper with appendix figures","","This paper evaluates whether welfare reform affects earnings solely through mean impacts that are constant within subgroups yet vary across them. The analysis targets a common practice in treatment-effect heterogeneity research that restricts attention to mean impacts varying only by subgroup. Using a novel simulation approach for treated-group earnings under the constant mean-impacts-within-subgroup model, the study finds poor capture of Jobs First treatment heterogeneity in Connecticut, using quantile treatment effects. It also shows that excluding within-group heterogeneity would conceal results consistent with core labor supply theory predictions.",{"@graph":14,"@context":73},[15,34,56],{"@type":16,"itemListElement":17},"BreadcrumbList",[18,23,27,31],{"item":19,"name":20,"@type":21,"position":22},"https://docshare.wps.com","Home","ListItem",1,{"item":24,"name":25,"@type":21,"position":26},"https://docshare.wps.com/document/","Document",2,{"item":28,"name":29,"@type":21,"position":30},"https://docshare.wps.com/document/research-report/","Research & Report",3,{"item":32,"name":10,"@type":21,"position":33},"https://docshare.wps.com/document/can-variation-in-subgroups-average-treatment-effects-explain-treatment-effect-heterogeneity-evidence-from-a-social-experiment-draft-paper-with-appendix-figures/201891/",4,{"url":32,"name":10,"@type":35,"image":36,"author":41,"headline":10,"publisher":44,"fileFormat":47,"inLanguage":8,"description":12,"dateModified":48,"datePublished":49,"encodingFormat":47,"isAccessibleForFree":50,"interactionStatistic":51},"DigitalDocument",{"url":37,"@type":38,"width":39,"height":40},"https://docshare.wps.com/thumbnails/can-variation-in-subgroups-average-treatment-effects-explain-treatment-effect-heterogeneity-evidence-from-a-social-experiment-draft-paper-with-appendix-figures/201891.png","ImageObject",300,407,{"name":42,"@type":43},"Miles","Person",{"url":19,"name":45,"@type":46},"DocShare","Organization","application/pdf","2026-11-01","2026-09-04",true,{"@type":52,"interactionType":53,"userInteractionCount":55},"InteractionCounter",{"@type":54},"ViewAction",14,{"@type":57,"mainEntity":58},"FAQPage",[59,65,69],{"name":60,"@type":61,"acceptedAnswer":62},"What question does the paper investigate about welfare reform effects?","Question",{"text":63,"@type":64},"It assesses whether welfare reform changes earnings only through subgroup mean impacts that are constant within subgroups but vary across subgroups.","Answer",{"name":66,"@type":61,"acceptedAnswer":67},"How does the paper’s approach to treatment-effect heterogeneity differ from the conventional one?",{"text":68,"@type":64},"The conventional approach estimates mean treatment effects that vary across subgroups using covariates. This paper instead simulates treated earnings under a “mean-impacts-only” within-subgroup model and evaluates how well it reproduces heterogeneity using quantile treatment effects.",{"name":70,"@type":61,"acceptedAnswer":71},"What do the findings say about the Jobs First experiment?",{"text":72,"@type":64},"The constant mean-impacts-within-subgroup model performs poorly at capturing treatment-effect heterogeneity for Connecticut’s Jobs First experiment, and ignoring within-group heterogeneity would miss evidence aligned with basic labor supply theory predictions.","https://schema.org",{"og:url":32,"og:type":75,"og:title":10,"og:site_name":45,"og:description":12},"article",{"robots":77,"canonical":32},"index,follow",{"doc_id":79,"site_id":7},201891,1788539865,{"code":4,"msg":82,"data":83},"success",[84,88,92,96,101,106,111,115,120,123,127],{"id":22,"doc_module":4,"doc_module_name":25,"category_name":85,"show_sort_weight":86,"slug":87},"Story & Novel",90,"story-novel",{"id":26,"doc_module":4,"doc_module_name":25,"category_name":89,"show_sort_weight":90,"slug":91},"Literature",80,"literature",{"id":33,"doc_module":4,"doc_module_name":25,"category_name":93,"show_sort_weight":94,"slug":95},"Exam",70,"exam",{"id":97,"doc_module":4,"doc_module_name":25,"category_name":98,"show_sort_weight":99,"slug":100},5,"Comic",60,"comic",{"id":102,"doc_module":4,"doc_module_name":25,"category_name":103,"show_sort_weight":104,"slug":105},6,"Technology",50,"technology",{"id":107,"doc_module":4,"doc_module_name":25,"category_name":108,"show_sort_weight":109,"slug":110},7,"Healthcare",40,"healthcare",{"id":112,"doc_module":4,"doc_module_name":25,"category_name":29,"show_sort_weight":113,"slug":114},8,30,"research-report",{"id":116,"doc_module":4,"doc_module_name":25,"category_name":117,"show_sort_weight":118,"slug":119},9,"Religion & Spirituality",20,"religion-spirituality",{"id":118,"doc_module":4,"doc_module_name":25,"category_name":121,"show_sort_weight":118,"slug":122},"World Cup","world-cup",{"id":124,"doc_module":4,"doc_module_name":25,"category_name":125,"show_sort_weight":124,"slug":126},10,"Lifestyle","lifestyle",{"id":128,"doc_module":4,"doc_module_name":25,"category_name":129,"show_sort_weight":97,"slug":130},19,"General","general",{"code":4,"msg":82,"data":132},{"doc_id":79,"user_id":133,"nickname":42,"user_avatar":134,"doc_module":4,"category_id":112,"category_name":29,"doc_title":10,"doc_description":12,"doc_content":135,"file_id":136,"file_url":137,"file_type":138,"file_size":139,"view_count":55,"is_deleted":4,"is_public":22,"is_downloadable":22,"audit_status":22,"page_count":140,"language":141,"language_code":8,"site_id":7,"html_lang":8,"table_of_contents":142,"faqs":143,"seo_title":144,"seo_description":12,"update_tm":80,"read_time":145,"preview_page_count":4},13056703019404,"https://ap-avatar.wpscdn.com/davatar_29158cc5080c5b710cf443261637dec0","Can Variation in Subgroups’ Average Treatment Effects Explain Treatment Effect Heterogeneity? Evidence from a Social  \nExperiment  \nMarianne P. Bitler  \nUniversity of California, Irvine and NBER  \nJonah B. Gelbach  \nUniversity of Pennsylvania Law School  \nHilary W. Hoynes  \nUniversity of California, Berkeley and NBER 1  \nThis version: April 2014  \n1 Correspondence to Hoynes at Richard & Rhoda Goldman School of Public Policy, UC Berkeley, 2607 Hearst Avenue, Berkeley, CA 94720-7320, phone (510) 642-1166, fax (510) 643-9657, [or](or hoynes@berkeley.edu)[ h](or hoynes@berkeley.edu)[oynes@berkeley.edu](or hoynes@berkeley.edu); Gelbach at [j](jgelbach@law.upenn.edu)[gelbach@law.upenn.edu](jgelbach@law.upenn.edu); or Bitler at [m](mbitler@uci.edu. The)[bitler@uci.edu. The](mbitler@uci.edu. The) data used in this paper are derived  \nfrom data files made available to researchers by MDRC. The authors remain solely responsible for how the data have been used or interpreted. We are very grateful to MDRC for providing the public access to the experimental data used here. We would also like to thank Alberto Abadie, Michael Anderson, Joe Altonji, Richard Blundell, Mike Boozer, David Brownstone, Moshe Buchinsky, Raj Chetty, Julie Cullen, Joe Cummins, Peng Ding, Avi Feller, David Green, Jeff Grogger, Jon Guryan, John Ham, Pat Kline, Thomas Lemieux, Bruce Meyer, Luke Miratrix, Robert Moffitt, Enrico Moretti, Giuseppe Ragusa, Shu Shen, Jeff Smith, Melissa Tartari, and Rob Valetta for helpful conversations, as well as seminar participants at the IRP Summer Research Workshop, the SOLE meetings, the IZA-SPEAC conference, the Harris School, UBC, UC Davis, UC Irvine, UCL, UCLA, UCSD, UCSC, UCSB, the San Francisco Federal Reserve Bank, Tinbergen Institute, Toronto, and Yale University.  \nAbstract  \nIn this paper, we assess whether welfare reform affects earnings only through mean impacts that are constant within but vary across subgroups. This is important because researchers interested in treatment effect heterogeneity typically restrict their attention to estimating mean impacts that are only allowed to vary across subgroups. Using a novel approach to simulating treatment group earnings under the constant mean-impacts within subgroup model, we find that this model does a poor job of capturing the treatment effect heterogeneity for Connecticut’s Jobs First welfare reform experiment using quantile treatment effects. Notably, ignoring within-group heterogeneity would lead one to miss evidence that the Jobs First experiment’s effects are consistent with central predictions of basic labor supply theory.  \n1 Introduction  \nIn previous work we estimated quantile treatment effects using data from a randomized experiment to evaluate the labor supply impact of Connecticut’s welfare reform program Jobs First (Bitler, Gelbach & Hoynes (2006)) . In that context, labor supply theory predicts that the reform should cause heterogeneous treatment effects across the earnings distribution. The data revealed a pattern consistent with these predictions, which, roughly speaking, are that there should be mass points at zero earnings in both the treated and control distributions, positive earnings effects in the middle of the earnings distribution, and negative earnings effects at the top of the earnings distribution. We found exactly this pattern of results, as Figure 3 of our earlier paper shows, using estimated quantile treatment effects (QTE) of Jobs First on the earnings distribution. Moreover, the range of QTE was quite broad, from-$300 to $500, far above the mean impact of $82 .  \nOur QTE-based approach to measuring treatment effect heterogeneity differs from the conventional one. One common approach involves estimating mean treatment effects but allowing the treatment effects to vary across subgroups based on demographic or other covariates. One then evaluates whether the subgroup-specific differences in treatment effects appear to vary importantly (for a review within the welfare reform ","cbCaipAz5aOb5F3O","https://ap.wps.com/l/cbCaipAz5aOb5F3O","pdf",905272,35,"English","# Abstract\n# 1 Introduction\n## Background on quantile treatment effects\n## Conventional mean-impact-by-subgroup approach\n## Proposed simulated earnings under the null model","[{\"question\":\"What question does the paper investigate about welfare reform effects?\",\"answer\":\"It assesses whether welfare reform changes earnings only through subgroup mean impacts that are constant within subgroups but vary across subgroups.\"},{\"question\":\"How does the paper’s approach to treatment-effect heterogeneity differ from the conventional one?\",\"answer\":\"The conventional approach estimates mean treatment effects that vary across subgroups using covariates. This paper instead simulates treated earnings under a “mean-impacts-only” within-subgroup model and evaluates how well it reproduces heterogeneity using quantile treatment effects.\"},{\"question\":\"What do the findings say about the Jobs First experiment?\",\"answer\":\"The constant mean-impacts-within-subgroup model performs poorly at capturing treatment-effect heterogeneity for Connecticut’s Jobs First experiment, and ignoring within-group heterogeneity would miss evidence aligned with basic labor supply theory predictions.\"}]","Can Variation in Subgroups’ Average Treatment Effects Explain Treatment Effect Heterogeneity? Evidence from a Social Experiment - Draft paper with appendix figures | PDF",88]