The Interplay of Optimization and Machine Learning to Solve Large-scale Black-box Noisy Functions - dissertation

Aug 4, 2026

High-dimensional black-box optimization poses a persistent barrier in modern science and engineering. This dissertation develops an adaptive interplay between optimization and machine learning, using surrogate models to balance exploration, exploitation, and estimation. A multi-level Partitioning and Branch-and-Bound algorithm for level-set approximation is improved with importance sampling, Gaussian-process guidance, and adaptive sampling probabilities supported by confidence intervals. To address high dimensionality, it introduces BASSO with finite-time analysis and surrogate-driven sampling, and extends a noisy optimization scheme via quadratic regression and optimistic sampling with convergence and numerical validation.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages160

Reading Time6 h 43 min

AccessFree to download

Abstract

Multi-level PBnB for level-set approximation

BASSO framework and scalability

Noisy optimization via extended SOSA

©Copyright 2025 Pariyakorn Maneekul The Interplay of Optimization and Machine Learning to Solve Large-scale Black-box Noisy Functions Pariyakorn Maneekul A dissertation submitted in partial fulﬁllment of the requirements for the degree of Doctor of Philosophy University of Washington 2025 Reading Committee: Zelda B. Zabinsky, Chair Chiwoo Park Giulia Pedrielli Program Authorized to Offer Degree: Industrial and Systems Engineering University of Washington Abstract The Interplay of Optimization and Machine Learning to Solve Large-scale Black-box Noisy Functions Pariyakorn Maneekul Chair of the Supervisory Committee: Zelda B. Zabinsky Industrial and Systems Engineering High-dimensional black-box optimization presents an increasingly prevalent challenge in modern science and engineering. This dissertation addresses this challenge through a novel interplay between optimization and machine learning methods, developing adaptive search algorithms that strike a balance between exploration, exploitation, and estimation. The proposed algorithms leverage machine learning techniques to construct surrogate models, thereby enhancing the efﬁciency of the optimization process. The dissertation proposes a multi-level Partitioning and Branch-and-Bound (PBnB) algorithm designed for level-set approximation, enhancing the original PBnB algorithm signiﬁcantly. This multi-level PBnB algorithm employs importance sampling to strategically identify promising subregions of the partitioned search space. Its performance is further enhanced by integrating Gaussian processes as a surrogate model to guide local sampling exploration. During the process, the target level set is approximated by classifying subregions as either pruned (no intersection with target level set), maintained (contained within target level set), or undecided. This enhanced version of the PBnB algorithm introduces an adaptive sampling probability that strategically directs samples to the most promising regions. Since this importance sampling results in dependency amongst samples, we have applied a statistical method to construct a conﬁdence interval on the probability of correctly classifying a subregion as pruned or maintained. The contribution to the interplay of optimization and machine learning is the local sampling within each subregion. We incorporate Gaussian processes and regularized quadratic regression, common and successful methods for prediction in machine learning for level-set approximation. The analysis of this multi-level PBnB algorithm quantiﬁes the quality of the level set approximation by deriving probability bounds on the volume of incorrectly pruned or maintained regions, which accounts for the effects of importance sampling. To address the challenges of high dimensionality, this dissertation introduces the Branching Adaptive Surrogate Search Optimization (BASSO) framework that conceptualizes the use of branching and surrogate modeling for black-box optimization. BASSO generalizes multi-level PBnB and adapts it to optimization as opposed to level-set approximation. A ﬁnite-time analysis of BASSO proves that the expected number of BASSO function evaluations needed to ﬁrst sample a point in the global optimum vicinity is linear in dimension given that two strong assumptions are satisﬁed. The desired linearity result suggests an algorithm that is scalable to high dimensions in theory. This research explores several variations to implement BASSO and partially satisfy the two assumptions. In this part of the research, methods used in machine learning are introduced to improve the chance of sampling in the improving region. One BASSO implementation incorporates Gaussian processes as a surrogate model and a second uses regularized quadratic regression as a surrogate model to predict where to sample next within a subregion. The synergy between the surrogate model and the optimization algorithm work together to balance explo

FAQs

What problem does the dissertation address?

It addresses high-dimensional black-box optimization, where function evaluations are expensive and only limited information is available.

How does the work combine optimization with machine learning?

It uses machine learning techniques to build surrogate models that guide search decisions, improving efficiency while balancing exploration and exploitation.

What key algorithms and extensions are proposed?

It proposes a multi-level PBnB algorithm for level-set approximation and a BASSO framework for generalized branching-and-surrogate search, and it extends SOSA for noisy function estimation using quadratic regression and optimistic sampling.

Aug 4, 2026

Upload by Valentina