[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-123892-en":3,"doc-seo-123892-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},123892,137441390410,"Hazel","https://ap-avatar.wpscdn.com/avatar/2000252f4ab5702993?_k=1776741390130283984",8,"Research & Report","Recent Developments in Machine Learning Methods for Stochastic Control and Games","Stochastic optimal control and games underpin applications across finance, economics, social sciences, robotics, and energy management, where complex models demand advanced numerical methods. This review centers on deep learning approaches for solving stochastic control problems and games in continuous time and space. It highlights neural-network techniques for high-dimensional PDEs and backward stochastic differential equations, as well as model-free reinforcement learning for Markov decision processes. The work introduces these methods and summarizes state-of-the-art research at the intersection of machine learning, stochastic control, and games.","arXiv :2303 . 10257v3 [math .OC] 11 Mar 2024  \nRecent Developments in Machine Learning Methods for Stochastic Control and Games  \nRuimeng Hu 1 Mathieu Lauri`ere 2  \nAbstract  \nStochastic optimal control and games have a wide range of applications, from finance and economics to social sciences, robotics, and energy management. Many real-world applications involve complex models that have driven the development of sophisticated numerical methods. Recently, computational methods based on machine learning have been developed for solving stochastic control problems and games. In this review, we focus on deep learning methods that have unlocked the possibility of solving such problems, even in high dimensions or when the structure is very complex, beyond what traditional numerical methods can achieve. We consider mostly the continuous time and continuous space setting. Many of the new approaches build on recent neural-network-based methods for solving high-dimensional partial differential equations or backward stochastic differential equations, or on model-free reinforcement learning for Markov decision processes that have led to breakthrough results. This paper provides an introduction to these methods and summarizes the state-of-the-art works atthe crossroad of machine learning and stochastic control and games.  \nContents  \n1 Introduction 2  \n1.1 Some high-dimensional examples in applications ........................ 3  \n1.2 An illustrative linear quadratic model .............................. 4  \n1.3 Organization of the survey ..................................... 6  \n2 Stochastic Control Problems 7  \n2.1 Formulation of stochastic control ................................. 7  \n2.2 Direct parameterization ...................................... 10  \n2.2.1 Global in time approach .................................. 10  \n2.2.2 Local in time approach .................................. 11  \n2.3 BSDE-based deep learning algorithms .............................. 12  \n2.3.1 Deep backward stochastic differential equation (Deep BSDE) method ........ 12  \n2.3.2 Deep backward dynamic programming (DBDP) ..................... 13  \n2.4 Primal-Dual approaches ...................................... 14  \n2.5 PDE-based algorithms ....................................... 15  \n2.6 Extensions .............................................. 16  \n2.6.1 Stochastic control with delay ............................... 16  \n2.6.2 Mean-field type control .................................. 19  \n1 Department of Mathematics, and Department of Statistics and Applied Probability, University of California, Santa Barbara, CA 93106-3080, USA, [rhu@ucsb.edu](rhu@ucsb.edu).  \n2 Shanghai Frontiers Science Center of Artificial Intelligence and Deep Learning; NYU-ECNU Institute of Mathematical Sciences, NYU Shanghai, 567 West Yangsi Road, Shanghai, 200126, People’s Republic of China, [mathieu.lauriere@nyu.edu](mathieu.lauriere@nyu.edu). Corresponding author.  \nKeywords: Stochastic optimal control, stochastic games, mean field games machine learning, deep learning  \nMSC Numbers: 49N70, 49N80, 68T07  \n3 Stochastic Differential Games 22  \n3.1 N-player stochastic games ..................................... 23  \n3.1.1 Theoretical background .................................. 23  \n3.1.2 Direct parameterization .................................. 25  \n3.1.3 BSDE-based deep learning algorithms .......................... 27  \n3.1.4 PDE-based deep learning algorithms ........................... 32  \n3.2 Mean-field games .......................................... 32  \n3.2.1 Theoretical background .................................. 33  \n3.2.2 Direct parameterization .................................. 38  \n3.2.3 BSDE-based deep learning algorithms .......................... 41  \n3.2.4 PDE-based deep learning algorithms ........................... 43  \n4 Reinforcement Learning 49  \n4.1 Reinforcement learning for stochastic control problems ..................... 50  \n4.1.1 Markov decision proc","cbCaivfPFJ0gNDrF","https://ap.wps.com/l/cbCaivfPFJ0gNDrF","pdf",8043404,1,84,"English","en",105,"# Introduction\n## Some high-dimensional examples in applications\n## An illustrative linear quadratic model\n## Organization of the survey\n# Stochastic Control Problems\n## Formulation of stochastic control\n## Direct parameterization\n## BSDE-based deep learning algorithms\n## Primal-Dual approaches\n## PDE-based algorithms\n## Extensions\n# Stochastic Differential Games\n## N-player stochastic games\n## Mean-field games\n# Reinforcement Learning\n## Reinforcement learning for stochastic control problems\n## Reinforcement learning for stochastic differential games\n# Conclusion and Future Directions","[{\"question\":\"What problem area does the review focus on?\",\"answer\":\"It focuses on stochastic optimal control and stochastic differential games, especially computational methods for solving them.\"},{\"question\":\"Which machine learning approaches are emphasized?\",\"answer\":\"The review emphasizes deep learning methods for high-dimensional PDEs and backward stochastic differential equations, and also model-free reinforcement learning for Markov decision processes.\"},{\"question\":\"Why is deep learning useful for these stochastic problems?\",\"answer\":\"Deep learning methods enable solving tasks in high dimensions or with complex structures beyond what traditional numerical methods can achieve.\"}]","Recent Developments in Machine Learning Methods for Stochastic Control and Games | PDF",1785819106,212,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"recent-developments-in-machine-learning-methods-for-stochastic-control-and-games","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/recent-developments-in-machine-learning-methods-for-stochastic-control-and-games/123892/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-04",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem area does the review focus on?","Question",{"text":75,"@type":76},"It focuses on stochastic optimal control and stochastic differential games, especially computational methods for solving them.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"Which machine learning approaches are emphasized?",{"text":80,"@type":76},"The review emphasizes deep learning methods for high-dimensional PDEs and backward stochastic differential equations, and also model-free reinforcement learning for Markov decision processes.",{"name":82,"@type":73,"acceptedAnswer":83},"Why is deep learning useful for these stochastic problems?",{"text":84,"@type":76},"Deep learning methods enable solving tasks in high dimensions or with complex structures beyond what traditional numerical methods can achieve.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]