Auto-FL-Research: Agentic Search for Federated Learning Algorithms

Jul 16, 2026

Upload by Hazel

Federated learning research depends on many intertwined algorithmic design choices—optimizer variants, server aggregation rules, client update schedules, normalization, regularization, and model architecture. Auto-FL-Research (AFR) introduces a constrained coding-agent workflow that searches for FL “recipe” candidates while restricting code edits to a task-defined mutation surface. Fixed execution and evaluation contracts ensure fair comparisons and record budgets, scores, artifacts, and failures. Evaluations on FLamby and LEAF show mixed transfer, seed sensitivity, and repeat/held-out robustness effects.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages8

Reading Time20 min

Introduction

Related Work

Federated Optimization

Federated Hyperparameter Optimization

Auto-FL-Research: Agentic Search for Federated Learning Algorithms Holger R. Roth, Ziyue Xu, Chester Chen, Daguang Xu, Peter Cnudde, Andrew Feng NVIDIA, Santa Clara, USA arXiv :2607 .0 1366v 1 [ cs .AI] 1 Jul 2026 Abstract—Federated learning (FL) research often depends on many small but consequential algorithmic choices: optimizer variants, server aggregation rules, local training schedules, normalization, regularization, and model architecture. These choices are expensive to explore manually and difficult to compare fairly when candidate changes can also alter the FL training or evaluation path. In this work, we present Auto-FL-Research (AFR), a constrained coding-agent workflow for FL algorithmic recipe search. Agents may propose and implement candidate training algorithms, including server aggregation rules, client update schedules, local objectives, and registered model variants, while task profiles fix the mutation surface, compute budget, communication contract, and final model evaluation. Each campaign records candidate scores, runtime, edited files, artifacts, and failure status. We evaluate AFR on five healthcare cross-silo FLamby tasks and on grouped-client profiles for the five fixed LEAF datasets plus the LEAF synthetic task. Five-seed repeat evaluations support gains on four FLamby tasks and five of six LEAF profiles, while also exposing seed-sensitive and search-selected failure cases. Same-budget controls show that several gains correspond to FL-recipe changes, whereas other improvements are recovered by fixed-surface scalar controls or fail under repeat or held-out evaluation. These mixed outcomes are part of the contribution: they show how agent-generated candidates can be separated into repeated FL mechanisms, fixed-surface tuning effects, and selected single-run artifacts. Index Terms—Federated Learning, Autonomous Agents, Hyperparameter Optimization, AutoML, NVIDIA FLARE, FLamby, LEAF I. INTRODUCTION Federated learning (FL) promises collaborative model development without centralizing raw data, but the practical performance of an FL system depends on a large design surface [1]. A practitioner must choose local optimizers, server aggregation rules, schedules, regularization, client participation, model architecture, evaluation strategy, and many task-specific details [2]. These choices interact with data heterogeneity and communication constraints, so improvements that appear obvious in centralized training can fail in FL. Automated FL methods have explored specific portions ofthis surface, including learnable aggregation, federated hyperparameter optimization, federated neural architecture search, and adaptive server optimizers [3]–[6]. However, many useful research advances are not a single scalar hyperparameter. A competitive FL algorithm may require introducing a new model architecture, changing a local loss, adding a server optimizer, or using an improved server aggregation method while preserving the protocol and the benchmark definition. Recent coding agents make it possible to automate code-level research loops, but unconstrained experimentation can confound evaluation: an agent can change the metric, alter the data split, silently increase compute, or break the FL contract. Auto-FLResearch addresses this by fixing what the agent may edit and how every candidate is evaluated. The agent is instructed and validated to Cross-s ite score 0.92 0.90 0.88 0.86 0.84 0 60 120 180 240 Experiment \# Fig. 1. Illustrative CIFAR-10 Auto-FL-Research campaign progress. Each point is a candidate in the run log; gray points are discarded candidates, blue points are active candidates, green points are kept candidates, and the green step line tracks the running best final global-model score. Purple markers indicate logged literature-review events. modify code only inside a task-defined mutation surface and must evaluate candidates through a fixed FL harness, here implemented with NV

FAQs

What problem does Auto-FL-Research (AFR) address in federated learning algorithm search?

AFR targets the difficulty of exploring many consequential FL design choices while ensuring candidates can be compared fairly and evaluated consistently under controlled execution and evaluation rules.

How does AFR prevent agent experiments from confounding evaluation results?

AFR fixes what the agent may edit via a task-defined mutation surface and evaluates each candidate through a fixed FL harness with recorded budgets, scores, artifacts, and failure status.

What do the evaluations on FLamby and LEAF reveal about the search results?

Results show gains on selected tasks and profiles, but also seed-sensitive failures and improvements that sometimes correspond to specific FL-recipe changes, while others arise from fixed-surface scalar tuning or do not survive repeated or held-out evaluation.

Jul 16, 2026

Upload by Hazel