Machine Learning Methods for Large Population Games with Applications in Operations Research

Aug 3, 2026

Upload by Riley

This tutorial introduces machine learning methods for computing Nash equilibria in games with a large number of agents, motivated by operations research applications such as epidemic control, optimal decisions in financial markets, electricity grid management, and traffic control for self-driving cars. It first reviews stochastic optimal control for a single agent in both discrete and continuous time. It then develops finite-agent dynamic games and the mean field games framework for efficient approximate Nash equilibrium computation. Finally, it presents reinforcement-learning and fixed-point approaches for discrete-time settings, and deep learning methods grounded in stochastic or partial differential equations for continuous-time problems, with examples and numerical illustrations throughout.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages39

Reading Time1 h 38 min

AccessFree to download

Introduction

Finite player games and mean field games

Background on single-agent control

Nash equilibria in finite player games

From large populations to mean field games

Examples and extensions

Discrete time models

Continuous time models

Extensions

Methods for discrete time models

Fixed point algorithms

Computing a best response by policy optimization

Computing a best response by dynamic programming

Numerical illustrations

Methods for continuous time models

Deep learning methods for solving FBSDEs

Deep learning methods for solving PDEs

Conclusion and perspectives

arXiv :2406 . 10441v1 [math .OC] 14 Jun 2024 Machine Learning Methods for Large Population Games with Applications in Operations Research G¨ok¸ce Dayanıklı∗ Mathieu Lauri`ere† Abstract In this tutorial, we provide an introduction to machine learning methods for finding Nashequilibria in games with large number of agents. These types of problems are important for the operations research community because of their applicability to real life situations such as control of epidemics, optimal decisions in financial markets, electricity grid management, or traffic control for self-driving cars. We start the tutorial by introducing stochastic optimal control problems for a single agent, in discrete time and in continuous time. Then, we present the framework of dynamic games with finite number of agents. To tackle games with a very large number of agents, we discuss the paradigm of mean field games, which provides an efficient way to compute approximate Nash equilibria. Based on this approach, we discuss machine learning algorithms for such problems. First in the context of discrete time games, we introduce fixed point based methods and related methods based on reinforcement learning. Second, we discuss machine learning methods that are specific to continuous time problems, by building on optimality conditions phrased in terms of stochastic or partial differential equations. Several examples and numerical illustrations of problems arising in operations research are provided along the way. Keywords. game theory; multi-agent systems; mean field games; machine learning; artificial intelligence; deep learning; reinforcement learning Contents 1 Introduction 2 2 Finite player games and mean field games 3 2.1 Background on single-agent control ............................... 4 2.2 Nash equilibria in finite player games .............................. 5 2.3 From large populations to mean field games .......................... 6 ∗ Department of Statistics, University of Illinois at Urbana-Champaign, Champaign, IL 61820, USA[gokced@illinois.edu](gokced@illinois.edu). †Shanghai Frontiers Science Center of Artificial Intelligence and Deep Learning; NYU-ECNU Institute of Mathematical Sciences, NYU Shanghai, 567 West Yangsi Road, Shanghai, 200126, People’s Republic of China, [mathieu.lauriere@nyu.edu](mathieu.lauriere@nyu.edu). ML for Large Population Games in OR 3 Examples and extensions 9 3.1 Discrete time models ....................................... 9 3.1.1 Crowd motion ...................................... 10 3.1.2 Traffic routing ...................................... 10 3.1.3 Cybersecurity ....................................... 11 3.2 Continuous time models ..................................... 12 3.2.1 Project value management ................................ 12 3.2.2 Electricity production .................................. 12 3.2.3 Trading with price impact ................................ 13 3.3 Extensions ............................................. 14 4 Methods for discrete time models 14 4.1 Fixed point algorithms ...................................... 15 4.2 Computing a best response by policy optimization ...................... 16 4.3 Computing a best response by dynamic programming .................... 17 4.4 Numerical illustrations ...................................... 19 4.4.1 Traffic routing with online mirror descent ....................... 19 4.4.2 Crowd motion with fictitious play and RL ....................... 20 5 Methods for continuous time models 23 5.1 Deep learning methods for solving FBSDEs .......................... 23 5.1.1 Algorithm 1: Iterative learning ............................. 24 5.1.2 Algorithm 2: Simultaneous learning .......................... 25 5.1.3 Numerical illustration: Epidemic model with FBSDE deep learning ......... 27 5.2 Deep learning methods for solving PDEs ............................ 28 5.2.1 Deep Galerkin method

FAQs

What problem does the tutorial address in large population games?

It focuses on finding Nash equilibria in games with many agents, using machine learning methods suited to large-scale multi-agent settings.

How does the tutorial connect operations research applications to the game-theoretic framework?

It explains that large-population games appear in real-world OR problems like epidemic control, financial market decisions, electricity grid management, and traffic control for self-driving cars.

What are the main machine learning approaches for discrete-time vs. continuous-time games?

For discrete time, it discusses fixed-point methods and reinforcement learning based policy/dynamic-programming style approaches. For continuous time, it presents deep learning methods that solve formulations expressed via stochastic equations or partial differential equations.

Aug 3, 2026

Upload by Riley