[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-123373-en":3,"doc-seo-123373-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},123373,962075114101,"Seraphina","https://ap-avatar.wpscdn.com/avatar/e000253a75eb197efd?x-image-process=image/resize,m_fixed,w_180,h_180&k=1780044092746381165",8,"Research & Report","A Two-Part Machine Learning Approach to Characterizing Network Interference in A/B Testing","Controlled experiments, commonly known as A/B tests, can be undermined by network interference, where units’ outcomes depend on others through their modeled network connections. This work addresses two core gaps: capturing complex social network structures and characterizing the resulting interference. A machine learning method is proposed using causal network motifs and transparent models to learn interference patterns from networked A/B data. Simulations and a large-scale Instagram test demonstrate improved performance over design-based cluster randomization and exposure mapping.","arXiv :2308 .09790v2 [ stat .ML] 29 Jun 2024  \nA Two-Part Machine Learning Approach to Characterizing Network Interference in A/B Testing  \nYuan Yuan  \nGraduate School of Management, University of California, Davis, CA, 95616  \nKristen M. Altenburger  \nCentral Applied Science, Meta Inc., Menlo Park, CA, 94025  \nThe reliability of controlled experiments, commonly referred to as “A/B tests,” is often compromised by network interference, where the outcomes of individual units are influenced by interactions with others. Significant challenges in this domain include the lack of accounting for complex social network structures and the difficulty in suitably characterizing network interference. To address these challenges, we propose a machine learning-based method. We introduce “causal network motifs” and utilize transparent machine learning models to characterize network interference patterns underlying an A/B test on networks. Our method’s performance has been demonstrated through simulations on both a synthetic experiment and a large-scale test on Instagram. Our experiments show that our approach outperforms conventional methods such as design-based cluster randomization and conventional analysis-based neighborhood exposure mapping. Our approach provides a comprehensive and automated solution to address network interference for A/B testing practitioners. This aids in informing strategic business decisions in areas such as marketing effectiveness and product customization.  \nKey words: experimental design, networks, interference, transparent machine learning, A/B testing  \n1 . Introduction  \nControlled experiments, also known as “A/B testing,” continue to serve as the cornerstone for making strategic decisions in business, including new product launches, marketing campaigns, and algorithm updates (Bakshy et al. 2014, Kohavi et al. 2020, Bojinov and Gupta 2022, Koning et al. 2022) . Through the random assignment of treatment or control groups, A/B testing facilitates the evaluation of causal, rather than merely correlational, impacts of a product intervention on business outcomes. Businesses have increasingly recognized the value of A/B testing and are investing in the development of in-house experimentation platforms (Kohavi et al. 2013, Bakshy et al. 2014, Xu et al. 2015) . Furthermore, numerous companies have adopted A/B testing software like Optimizely  \nand Split to efficiently perform and analyze their A/B tests. Companies adopting A/B testing have seen performance improvements of 30% to 100% within a year (Koning et al. 2022) .  \nHowever, a significant obstacle to the validity of A/B testing is network interference. Conventional causal inference rests on an essential assumption known as the “Stable Unit Treatment Value Assumption”(SUTVA) (Rubin 2005), which implies a unit’s outcome only depends on their treatment assignment. Network interference occurs when a unit’s (e.g., a person’s) outcome is influenced by the treatment assignments of other units, especially those within their network neighborhood, if their connections are modeled as a network (Hudgens and Halloran 2008, Toulis and Kao 2013, Basse and Airoldi 2018) . Network interference is prevalent in numerous contemporary A/B testing environments, including social media, online marketplaces, and location-based platforms (Hagiu and Wright 2015, Yan et al. 2018, Holtz et al. 2020, Li et al. 2022) . Failing to properly account for network interference is problematic. For instance, Holtz et al. (2020) found that network interference could skew the estimation of the treatment effect by more than 30% . Overall, network interference presents a significant challenge to industrial A/B tests, as it may mislead business decisions on product updates if the test results are unreliable.  \nEfforts to improve estimation in the presence of network interference have predominantly followed two research paths: pre-experimental design and post-experiment analysis (Eckles et al. 2016) . The","cbCaimNmWLtDn8ZD","https://ap.wps.com/l/cbCaimNmWLtDn8ZD","pdf",1374952,1,48,"English","en",105,"# Introduction\n## Network interference in A/B testing\n## Two research paths: design vs analysis\n## Exposure mapping framework (motif-based characterization)","[{\"question\":\"What problem does the document address in A/B testing?\",\"answer\":\"It addresses network interference, where a unit’s outcome is influenced by treatment assignments of other connected units, violating the usual stable unit assumption.\"},{\"question\":\"What is the proposed machine learning approach?\",\"answer\":\"The method uses causal network motifs and transparent machine learning models to characterize interference patterns underlying an A/B test on networks.\"},{\"question\":\"How is the approach evaluated and how does it compare to existing methods?\",\"answer\":\"It is evaluated through simulations on a synthetic experiment and a large-scale Instagram test, showing better performance than design-based cluster randomization and conventional exposure mapping methods.\"}]","A Two-Part Machine Learning Approach to Characterizing Network Interference in A/B Testing | PDF",1785816168,121,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"a-two-part-machine-learning-approach-to-characterizing-network-interference-in-ab-testing","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/a-two-part-machine-learning-approach-to-characterizing-network-interference-in-ab-testing/123373/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-04",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does the document address in A/B testing?","Question",{"text":75,"@type":76},"It addresses network interference, where a unit’s outcome is influenced by treatment assignments of other connected units, violating the usual stable unit assumption.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What is the proposed machine learning approach?",{"text":80,"@type":76},"The method uses causal network motifs and transparent machine learning models to characterize interference patterns underlying an A/B test on networks.",{"name":82,"@type":73,"acceptedAnswer":83},"How is the approach evaluated and how does it compare to existing methods?",{"text":84,"@type":76},"It is evaluated through simulations on a synthetic experiment and a large-scale Instagram test, showing better performance than design-based cluster randomization and conventional exposure mapping methods.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]