Distributional Robustness and Machine Learning Methods for Stochastic Decision-Making - Doctoral Thesis

Aug 3, 2026

Upload by Noah

Data-driven decision-making increasingly influences classification, prediction, optimization, and resource allocation, yet standard machine learning models struggle with uncertainty from future data or new application domains. This thesis studies how distributionally robust optimization can be integrated with modern learning to improve reliability and reduce performance risk under distribution shifts. It provides new theoretical results and strong experiments across three themes: generalization bounds for DRO, prediction-error queue scheduling, and tabular classification under |·| shifts, leveraging LLM embeddings to boost few-shot target performance.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages223

Reading Time9 h 22 min

AccessFree to download

Chapter 1: Introduction

Chapter 2: Generalization Power of Distributionally Robust Optimization: From Worst-Case Analysis to Minimal Hypothesis Class Dependence

2.2 Related Work and Comparisons

2.3 General Results

2.4 Specialization to MMD DRO

2.5 Specialization to 1-Wasserstein DRO

2.6 Specialization to f-divergence DRO

2.7 Extensions to Distributional Shift Settings

2.8 Numerical Experiments

Chapter 3: Design and Scheduling of an AI-based Queueing System

3.1 Introduction

3.2 Model

3.3 Lower bound on queueing cost

3.4 Heavy-traffic optimality of the P^2 φ-rule

3.6 Model selection based on queueing cost

3.7 Design of an AI-based triage system

3.8 Discussion

Chapter 4: LLM Embeddings Improve Test-time Adaptation to Tabular |·|-Shifts

4.1 Introduction

4.2 Methods

4.3 Numerical Experiments

4.4 Discussion

Distributional robustness and machine learning methods for stochastic decision-making Yibo Zeng Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy under the Executive Committee of the Graduate School of Arts and Sciences COLUMBIA UNIVERSITY © 2024 Yibo Zeng All Rights Reserved Abstract Distributional Robustness and Machine Learning Methods for Stochastic Decision-Making Yibo Zeng Recently, data-driven methods are increasingly shaping decision-making across domains such as classification, prediction, optimization, and resource allocation. While machine learning advancements have been pivotal, current models can be insufficient to handle unseen uncertainties from future data or new application domains, leading to unreliable decisions and performance risks. This thesis explores how distributionally robust methods can be combined with modern machine learning techniques to ensure more reliable decision-making. We develop new theoretical results and achieve state-of-the-art empirical results in three areas: generalization bounds in machine learning, queue scheduling with prediction errors, and tabular classification under 􀀮 | 􀀭 shifts. The work is organized into three chapters. In Chapter 2, we derive novel generalization bounds for distributionally robust optimization (DRO) . Our analysis implies generalization bounds whose dependence on the hypothesis class appears the minimal possible: The bound depends solely on the true loss function, independent of any other candidates in the hypothesis class. To our best knowledge, it is the first generalization bound of this type in the literature. Chapter 3 investigates optimal scheduling in service systems with prediction errors. We develop a near-optimal index-based policy that incorporates predicted class information. Our results guide model selection with a focus on downstream queueing performance and offer insights into designing queueing systems with AI-based triage. In Chapter 4, we study tabular data classification under 􀀮 | 􀀭 shift. We not only build a large-scale testbed, but also demonstrate that large language model (LLM) embeddings significantly improve classification performance, even with few labeled samples from the target domain. Table of Contents Acknowledgments ........................................ vii Dedication ............................................ viii Chapter 1: Introduction .................................... 1 Chapter 2: Generalization Power of Distributionally Robust Optimization: From Worst-Case Analysis to Minimal Hypothesis Class Dependence ............... 3 2.1 Introduction ...................................... 3 2.2 Related Work and Comparisons ............................ 7 2.2.1 Absolute Bounds on Expected Loss ..................... 7 2.2.2 Variability Regularization ........................... 9 2.2.3 Risk Aversion ................................. 11 2.3 General Results .................................... 11 2.3.1 A Beginning Bound .............................. 11 2.3.2 Specialization to G-IPM DRO ........................ 13 2.3.3 Strengths and Limitations Compared with ERM ............... 15 2.4 Specialization to MMD DRO ............................. 17 2.4.1 Comparison with [21] ............................. 18 2.4.2 Comparison with RKHS Regularized ERM ................. 19 2.4.3 Extension to ℓ( 􀀵 ∗ , ·) ∉ H ........................... 19 2.5 Specialization to 1-Wasserstein DRO ......................... 21 2.6 Specialization to 􀁱-divergence DRO ......................... 22 2.6.1 General Bounds for 􀁱-divergence DRO ................... 23 2.6.2 Specialization to Cressie-Read divergence .................. 24 2.6.3 Specialization to 􀁪2-divergence ....................... 26 2.7 Extensions to Distributional Shift Settings ...................... 27 2.8 Numerical Experiments ................................ 31 2.8.1 A Special Cas

FAQs

Why do current machine learning models struggle with stochastic decision-making under future uncertainty?

They may not adequately handle unseen uncertainties caused by future data or new application domains, which can lead to unreliable decisions and performance risk.

What is the key theoretical contribution of Chapter 2?

It derives distributionally robust optimization generalization bounds where dependence on the hypothesis class is minimized, relying solely on the true loss function.

How does Chapter 3 use prediction information for queue scheduling?

It develops a near-optimal index-based policy that incorporates predicted class information and guides model selection based on downstream queueing performance.

How do LLM embeddings help in Chapter 4 for tabular classification under distribution shifts?

They significantly improve classification performance on the target domain, even when only few labeled samples are available, enabling better test-time adaptation.

Aug 3, 2026

Upload by Noah