Anderson Acceleration and Dynamic Optimal Transport in Optimization - Theoretical Analysis, Algorithms, and Applications in Machine Learning - Thesis

Aug 3, 2026

Upload by Aria

This dissertation explores two core optimization topics: Anderson Acceleration and Dynamic Optimal Transport, covering theoretical analysis, algorithm design, and machine learning applications. It studies variants of Anderson Acceleration for nonlinear fixed-point problems and establishes equivalence with multisecant methods to clarify algorithm structure and support convergence analysis. It further develops the Alternating Anderson-Picard method, links it to multisecant-GMRES and Newton-GMRES, and applies a federated variant (FedOSAA) for decentralized learning. Finally, it integrates dynamic optimal transport into autoencoders to improve generative capability under limited data.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages125

Reading Time5 h 15 min

AccessFree to download

Contents

Chapter 1. Introduction

Chapter 2. Anderson Acceleration

Chapter 3. Alternating Anderson-Picard method

Chapter 4. Anderson Acceleration in Federated Learning

Chapter 5. Autoencoder and Dynamic Optimal Transport

Appendix A. Anderson Acceleration in Federated Learning

Appendix B. Autoencoder

UC Davis UC Davis Electronic Theses and Dissertations Title Anderson Acceleration and Dynamic Optimal Transport in Optimization: Theoretical Analysis, Algorithms, and Applications in Machine Learning Permalink [https://escholarship.org/uc/item/6kz7q0gh](https://escholarship.org/uc/item/6kz7q0gh) Author Feng, Xue Publication Date 2024 Peer reviewed|Thesis/dissertation [eScholarship.org](eScholarship.org) Powered by the California Digital Library University of California Anderson Acceleration and Dynamic Optimal Transport in Optimization: Theoretical Analysis, Algorithms, and Applications in Machine Learning By XUE FENG DISSERTATION Submitted in partial satisfaction of the requirements for the degree of DOCTOR OF PHILOSOPHY in APPLIED MATHEMATICS in the OFFICE OF GRADUATE STUDIES of the UNIVERSITY OF CALIFORNIA DAVIS Approved: | Thomas Strohmer, Chair | | --- | | Albert Fannjiang | Xin Liu Committee in Charge 2024 © First M. LastName, 2024 . All rights reserved. Contents Abstract iv Acknowledgments v Chapter 1 . Introduction 1 Chapter 2 . Anderson Acceleration 3 2.1. Algorithm, Notations and Preliminary Results 4 2.2. Equivalence between Anderson Acceleration and Multisecant Methods 15 2.3. Convergence Analysis 24 2.4. Conclusion 36 Chapter 3 . Alternating Anderson-Picard method 37 3.1. Algorithm, Notations and Preliminary Results 39 3.2. Equivalence between AAP(m) and Multisecant-GMRES Methods 41 3.3. Bound of Et 46 3.4. Optimization Gain Bound 54 3.5. Convergence Analysis 59 3.6. Numerical Results 63 3.7. Conclusion 65 Chapter 4 . Anderson Acceleration in Federated Learning 66 4.1. FedOSAA: Related Work and Algorithm 68 4.2. Convergence Analysis 71 4.3. Numerical Experiments 79 4.4. Conclusion 84 Chapter 5 . Autoencoder and Dynamic Optimal Transport 86 5.1. Background 89 5.2. Path Energy and Proposed Algorithm 91 5.3. Numerical Experiments 96 5.4. Conclusion 103 Appendix A. Anderson Acceleration in Federated Learning 105 A.1 . State-of-arts FL Algorithms 105 A.2 . GIANT Method 108 Appendix B. Autoencoder 109 B.1 . Additional Experiments 109 Bibliography 112 Abstract This dissertation explores two important areas in optimization: Anderson Acceleration (AA) and Dynamic Optimal Transport (OT), in terms of theoretical analysis, algorithmic design, and applications in machine learning. Part I discusses different variants of AA, such as AA with window mand restarted AA, to solve nonlinear fixed-point problems. The core is to discuss their equivalence with multisecant methods which not only help to understand the nature of the algorithms but also indicate the later convergence analysis. Part II introduces the Alternating Anderson-Picard (AAP) method, which combines the efficiency of AA in speeding up convergence and the simplicity of Picard iteration. The theoretical results demonstrate its equivalence to multisecant-GMRES methods and shows its deep connection to Newton-GMRES methods. Part III applies AA to federated learning and proposes the Federated One-Step Anderson Acceleration (FedOSAA) for effective decentralized machine learning. Part IV focuses on OT. The proposed algorithm integrates dynamic OT into the autoencoder to enhance its generative ability when data is limited. Overall, this thesis connects theoretical insights with practical applications and advances computational techniques in both scientific computing and machine learning. Acknowledgments I wrote this thesis acknowledgment after I caught a rat in my kitchen. My initial thought was to start it with a sad tone as my Ph.D. journey was long and lost. But it could be more interesting than that. Due to suspicious noises, I doubted the presence of a rat [in my kitchen. With my Ph.D. training](in my kitchen. With my Ph.D. training), I need to gather evidence to confirm my suspicions first. I set up a motion-detecting camera and, sure en

FAQs

What are the main research areas in this dissertation?

The dissertation focuses on Anderson Acceleration and Dynamic Optimal Transport, examining both theoretical aspects and practical algorithms in machine learning.

How does the Alternating Anderson-Picard (AAP) method relate to other methods?

AAP combines Anderson Acceleration and Picard iteration, and the theoretical results show equivalence to multisecant-GMRES while highlighting connections to Newton-GMRES methods.

What application does FedOSAA target?

FedOSAA is proposed for federated learning, enabling effective decentralized machine learning using a one-step Anderson acceleration idea.

Aug 3, 2026

Upload by Aria