[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-85501-en":3,"doc-seo-85501-105":29,"detail-sidebar-cat-0-en-105":90},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},85501,34359740700684,"Finn","https://ap-avatar.wpscdn.com/avatar/1f400023980c374ae676?_k=1777273430885731487",8,"Research & Report","Reinforcement Learning in the Real World: A Survey of Statistical Challenges and Future Directions","Reinforcement learning (RL) has delivered strong results in real-world decision-making across gaming, robotics, online advertising, public health, and natural language processing. Yet a persistent gap remains between RL research and practical deployment, driven by two core statistical challenges: limited interaction opportunities during learning and substantial environment changes that force redesign. The work proposes a three-component deployment framework—online optimization, post/between-deployment offline inference, and repeated deployment cycles—then reviews methods for sample efficiency, data utility, and continual improvement.","arXiv :2601 . 15353v2 [ stat .AP] 12 Jul 2026  \nReinforcement Learning in the Real World: A Survey of Statistical  \nChallenges and Future Directions  \nAsim H. Gazi 1 ,3 ,∗ , Yongyi Guo2 ,∗ , Daiqi Gao3 , Ziping Xu4 , Kelly W. Zhang5 , Susan A. Murphy 1 ,3  \n1 Department of Computer Science, Harvard University  \n2 Department of Statistics, University of Wisconsin–Madison  \n3 Department of Statistics, Harvard University  \n4 School of Data Science and Society, University of North Carolina at Chapel Hill  \n5 Department of Mathematics, Imperial College London  \n[agazi@g. harvard. edu](agazi@g. harvard. edu), [guo98@wisc. edu](guo98@wisc. edu), [daiqigao@gmail. com](daiqigao@gmail. com),  \n[zipingxu@unc. edu](zipingxu@unc. edu), [kelly. zhang@imperial. ac. uk](kelly. zhang@imperial. ac. uk), [samurphy@g. harvard. edu](samurphy@g. harvard. edu)  \n∗ Equal contribution  \nAbstract  \nReinforcement learning (RL) has achieved remarkable success in real-world decision-making across diverse domains, including gaming, robotics, online advertising, public health, and natural language processing. Despite these advances, a substantial gap remains between RL research and its deployment in many practical settings. Two recurring challenges often underlie this gap. First, many settings offer limited opportunity for the agent to interact extensively with the target environment due to practical constraints. Second, many target environments often undergo substantial changes, requiring redesign and redeployment of RL systems (e.g., advancementsin science and technology that change the landscape of healthcare delivery) . Addressing these challenges and bridging the gap between basic research and application requires theory and methodology that directly inform the design, implementation, and continual improvement of RL systems in real-world settings.  \nIn this paper, we frame the application of RL in practice as a three-component process: (i) online learning and optimization during deployment, (ii) post-or between-deployment offline analyses, and (iii) repeated cycles of deployment and redeployment to continually improve the RL system. We provide a narrative review of recent advances that address the statistical challenges arising across these three components, including methods for enhancing sample efficiency during online deployment, maximizing data utility for post- or between-deployment inference, and designing sequences of deployments for continual improvement. We also outline future research directions in RL that are use-inspired—aiming for impactful application of RL in practice.  \nKeywords: Reinforcement learning, adaptive experiments, adaptive interventions, online learning, statistical inference, sequential deployments  \n1 Introduction  \nReinforcement learning (RL) has received considerable interest in recent years due to superhuman or state-of-the-art performance in areas such as gaming, autonomous driving, and robotics [Mnih  \net al. , 2015 , Silver et al. , 2017 , Wurman et al. , 2022 , Tang et al. , 2025 , Kaufmann et al. , 2023] . One of the factors behind these real-world successes is the proliferation of deep learning techniques that enable flexible learning from big data. Common characteristics of applications where deep RL has success include (1) areas in which the RL algorithm can interact extensively with the target environment or a high-fidelity simulator, and (2) minimal changes in the environment’s key variablesand dynamics such that re-learning via online RL and/or continual redesign of the RL algorithm is often not required or only infrequently required for successful decision-making by the algorithm in future deployments. For example, a recent high-profile example of RL success in the real world is the use of deep RL to defeat champion human racers in Gran Turismo [Wurman et al. , 2022], a highly realistic automobile racing simulator. In this work, the authors train a deep RL agent entirely offline in simulation—using population","cbCaincrlnjcD5RQ","https://ap.wps.com/l/cbCaincrlnjcD5RQ","pdf",1574872,1,45,"English","en",105,"# Introduction\n## Two Real-World Challenges (C1 and C2)\n## Three-Component Deployment Process\n## Statistical Methods and Future Directions","[{\"question\":\"What two challenges create the research-to-deployment gap in real-world reinforcement learning?\",\"answer\":\"The survey highlights C1: limited ability to interact extensively with the environment or a high-fidelity simulator to collect data. It also emphasizes C2: significant environment changes that require updating the RL algorithm for future deployments.\"},{\"question\":\"How does the document structure reinforcement learning in practice?\",\"answer\":\"It frames practice as a three-component process: (i) online learning and optimization during deployment, (ii) post- or between-deployment offline analyses, and (iii) repeated cycles of deployment and redeployment to continually improve the system.\"},{\"question\":\"Why is human-in-the-loop reinforcement learning especially difficult?\",\"answer\":\"Exploration is constrained by safety and engagement concerns, since actions can harm people, burden them, or cause disengagement, while human needs and availability to interact can also change.\"}]",1784204036,113,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":85,"head_meta":87,"extra_data":89,"updated_unix":27},"reinforcement-learning-in-the-real-world-a-survey-of-statistical-challenges-and-future-directions","",{"@graph":35,"@context":84},[36,53,67],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/reinforcement-learning-in-the-real-world-a-survey-of-statistical-challenges-and-future-directions/85501/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":61,"encodingFormat":60,"isAccessibleForFree":62,"interactionStatistic":63},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-16",true,{"@type":64,"interactionType":65,"userInteractionCount":4},"InteractionCounter",{"@type":66},"ViewAction",{"@type":68,"mainEntity":69},"FAQPage",[70,76,80],{"name":71,"@type":72,"acceptedAnswer":73},"What two challenges create the research-to-deployment gap in real-world reinforcement learning?","Question",{"text":74,"@type":75},"The survey highlights C1: limited ability to interact extensively with the environment or a high-fidelity simulator to collect data. It also emphasizes C2: significant environment changes that require updating the RL algorithm for future deployments.","Answer",{"name":77,"@type":72,"acceptedAnswer":78},"How does the document structure reinforcement learning in practice?",{"text":79,"@type":75},"It frames practice as a three-component process: (i) online learning and optimization during deployment, (ii) post- or between-deployment offline analyses, and (iii) repeated cycles of deployment and redeployment to continually improve the system.",{"name":81,"@type":72,"acceptedAnswer":82},"Why is human-in-the-loop reinforcement learning especially difficult?",{"text":83,"@type":75},"Exploration is constrained by safety and engagement concerns, since actions can harm people, burden them, or cause disengagement, while human needs and availability to interact can also change.","https://schema.org",{"og:url":51,"og:type":86,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":88,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":91},[92,96,100,104,109,114,119,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":93,"show_sort_weight":94,"slug":95},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":97,"show_sort_weight":98,"slug":99},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":101,"show_sort_weight":102,"slug":103},"Exam",70,"exam",{"id":105,"doc_module":4,"doc_module_name":45,"category_name":106,"show_sort_weight":107,"slug":108},5,"Comic",60,"comic",{"id":110,"doc_module":4,"doc_module_name":45,"category_name":111,"show_sort_weight":112,"slug":113},6,"Technology",50,"technology",{"id":115,"doc_module":4,"doc_module_name":45,"category_name":116,"show_sort_weight":117,"slug":118},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":45,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":45,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":45,"category_name":136,"show_sort_weight":105,"slug":137},19,"General","general"]