Recent Developments in Machine Learning Methods for Stochastic Control and Games

Aug 4, 2026

Upload by Hazel

Stochastic optimal control and games underpin applications across finance, economics, social sciences, robotics, and energy management, where complex models demand advanced numerical methods. This review centers on deep learning approaches for solving stochastic control problems and games in continuous time and space. It highlights neural-network techniques for high-dimensional PDEs and backward stochastic differential equations, as well as model-free reinforcement learning for Markov decision processes. The work introduces these methods and summarizes state-of-the-art research at the intersection of machine learning, stochastic control, and games.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages84

Reading Time3 h 32 min

AccessFree to download

Introduction

Some high-dimensional examples in applications

An illustrative linear quadratic model

Organization of the survey

Stochastic Control Problems

Formulation of stochastic control

Direct parameterization

BSDE-based deep learning algorithms

Primal-Dual approaches

PDE-based algorithms

Extensions

Stochastic Differential Games

N-player stochastic games

Mean-field games

Reinforcement Learning

Reinforcement learning for stochastic control problems

Reinforcement learning for stochastic differential games

Conclusion and Future Directions

arXiv :2303 . 10257v3 [math .OC] 11 Mar 2024 Recent Developments in Machine Learning Methods for Stochastic Control and Games Ruimeng Hu 1 Mathieu Lauri`ere 2 Abstract Stochastic optimal control and games have a wide range of applications, from finance and economics to social sciences, robotics, and energy management. Many real-world applications involve complex models that have driven the development of sophisticated numerical methods. Recently, computational methods based on machine learning have been developed for solving stochastic control problems and games. In this review, we focus on deep learning methods that have unlocked the possibility of solving such problems, even in high dimensions or when the structure is very complex, beyond what traditional numerical methods can achieve. We consider mostly the continuous time and continuous space setting. Many of the new approaches build on recent neural-network-based methods for solving high-dimensional partial differential equations or backward stochastic differential equations, or on model-free reinforcement learning for Markov decision processes that have led to breakthrough results. This paper provides an introduction to these methods and summarizes the state-of-the-art works atthe crossroad of machine learning and stochastic control and games. Contents 1 Introduction 2 1.1 Some high-dimensional examples in applications ........................ 3 1.2 An illustrative linear quadratic model .............................. 4 1.3 Organization of the survey ..................................... 6 2 Stochastic Control Problems 7 2.1 Formulation of stochastic control ................................. 7 2.2 Direct parameterization ...................................... 10 2.2.1 Global in time approach .................................. 10 2.2.2 Local in time approach .................................. 11 2.3 BSDE-based deep learning algorithms .............................. 12 2.3.1 Deep backward stochastic differential equation (Deep BSDE) method ........ 12 2.3.2 Deep backward dynamic programming (DBDP) ..................... 13 2.4 Primal-Dual approaches ...................................... 14 2.5 PDE-based algorithms ....................................... 15 2.6 Extensions .............................................. 16 2.6.1 Stochastic control with delay ............................... 16 2.6.2 Mean-field type control .................................. 19 1 Department of Mathematics, and Department of Statistics and Applied Probability, University of California, Santa Barbara, CA 93106-3080, USA, [rhu@ucsb.edu](rhu@ucsb.edu). 2 Shanghai Frontiers Science Center of Artificial Intelligence and Deep Learning; NYU-ECNU Institute of Mathematical Sciences, NYU Shanghai, 567 West Yangsi Road, Shanghai, 200126, People’s Republic of China, [mathieu.lauriere@nyu.edu](mathieu.lauriere@nyu.edu). Corresponding author. Keywords: Stochastic optimal control, stochastic games, mean field games machine learning, deep learning MSC Numbers: 49N70, 49N80, 68T07 3 Stochastic Differential Games 22 3.1 N-player stochastic games ..................................... 23 3.1.1 Theoretical background .................................. 23 3.1.2 Direct parameterization .................................. 25 3.1.3 BSDE-based deep learning algorithms .......................... 27 3.1.4 PDE-based deep learning algorithms ........................... 32 3.2 Mean-field games .......................................... 32 3.2.1 Theoretical background .................................. 33 3.2.2 Direct parameterization .................................. 38 3.2.3 BSDE-based deep learning algorithms .......................... 41 3.2.4 PDE-based deep learning algorithms ........................... 43 4 Reinforcement Learning 49 4.1 Reinforcement learning for stochastic control problems ..................... 50 4.1.1 Markov decision proc

FAQs

What problem area does the review focus on?

It focuses on stochastic optimal control and stochastic differential games, especially computational methods for solving them.

Which machine learning approaches are emphasized?

The review emphasizes deep learning methods for high-dimensional PDEs and backward stochastic differential equations, and also model-free reinforcement learning for Markov decision processes.

Why is deep learning useful for these stochastic problems?

Deep learning methods enable solving tasks in high dimensions or with complex structures beyond what traditional numerical methods can achieve.

Aug 4, 2026

Upload by Hazel