FedCBO - Reaching Group Consensus in Clustered Federated Learning through Consensus-based Optimization

Aug 3, 2026

Upload by Liam

Federated learning integrates training across multiple users while protecting data privacy and respecting communication-loss constraints. Clustered federated learning additionally assumes an unknown group structure and aims to train useful models per group, not a single global model. The paper introduces FedCBO, a consensus-based optimization approach modeled with interacting particles independent of membership. A mean-field analysis characterizes the particle limit, and convergence guarantees establish simultaneous global optimization for general non-convex cluster losses in the mean-field regime.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages51

Reading Time2 h 9 min

AccessFree to download

Introduction

Federated learning and privacy/communication constraints

Clustered federated learning (CFL) and non-i.i.d. data

Problem setup: server, agents, unknown group partition

FedCBO overview and methodological motivation

Consensus-based optimization via interacting particles

Mean-field analysis and convergence guarantees

Experimental validation

Comparison to state-of-the-art methods

Efficacy and theoretical validation

FedCBO: Reaching Group Consensus in Clustered Federated Learning through Consensus-based Optimization Jos􀀓e A. Carrillo [carrillo@maths.ox.ac.uk](carrillo@maths.ox.ac.uk) Mathematical Institute University of Oxford Oxford OX2 6GG, UK Nicol􀀓as Garc􀀓􀀐a Trillos [garciatrillo@wisc.edu](garciatrillo@wisc.edu) Department of Statistics University of Wisconsin-Madison 1300 University Avenue, Madison, Wisconsin 53706, USA Sixu Li [sli739@wisc.edu](sli739@wisc.edu) Department of Statistics University of Wisconsin-Madison 1300 University Avenue, Madison, Wisconsin 53706, USA Yuhua Zhu [yuhua.zhu@stat.ucla.edu](yuhua.zhu@stat.ucla.edu) Department of Statistics and Data Science, University of California, Los Angeles Los Angeles, California 90095-1554, USA Editor: Qiang Liu Abstract Federated learning is an important framework in modern machine learning that seeks to integrate the training of learning models from multiple users, each user having their own local data set, in away that is sensitive to data privacy and to communication loss constraints. In clustered federated learning, one assumes an additional unknown group structure among users, and the goal is to train models that are useful for each group, rather than simply training a single global model for all users. In this paper, we propose a novel solution to the problem of clustered federated learning that is inspired by ideas in consensus-based optimization (CBO) . Our new CBO-type method is based on a system of interacting particles that is oblivious to group memberships. Our model is motivated by rigorous mathematical reasoning, which includes a mean-􀀌eld analysis describing the large number of particles limit of our particle system, as well as convergence guarantees for the simultaneous global optimization of general non-convex objective functions (corresponding to the loss functions of each cluster of users) in the mean-􀀌eld regime. Experimental results demonstrate the e􀀎cacy of our FedCBO algorithm compared to other state-of-the-art methods and help validate our methodological and theoretical work. Keywords: consensus-based optimization, clustered federated learning, interacting particle system, mean-􀀌eld limit, asymptotic convergence analysis. 1. Introduction The wide use of internet of things (IoT) devices in various applications such as home automation, personal health monitoring, and vehicle-to-vehicle communications has led to the generation of vast amounts of data across a collective of users. However, concerns around data privacy and security, as well as limitations on communication costs and bandwidth, have made it challenging for an 􀀍c2024 Jos􀀓e A. Carrillo, Nicol􀀓as Garc􀀓􀀐a Trillos, Sixu Li and Yuhua Zhu. License: CC-BY 4.0, see [https://creativecommons.org/licenses/by/4.0/](https://creativecommons.org/licenses/by/4.0/. Attribution)[. Attribution](https://creativecommons.org/licenses/by/4.0/. Attribution) requirements are provided at [http://jmlr.org/papers/v25/23-0764.html](http://jmlr.org/papers/v25/23-0764.html). Carrillo, Garca Trillos, Li and Zhu individual user to take advantage of this large amount of stored information. This has motivated the design and development of federated learning (FL) strategies, which aim at pooling information from learning models trained on local devices to build models without relying on the collection of local data (McMahan et al., 2017; Kairouz et al., 2021) . Standard FL approaches aim to learn one global model for all local clients/users (McMahanet al., 2017; Li et al., 2020; Mohri et al., 2019; Karimireddy et al., 2020) . However, data heterogeneity, also known as [non-i.i.d. data](non-i.i.d. data) setting, naturally arises in FL applications since data are usually generated from users' personal devices. Thus, it is expected that no single global model can perform well across all clients (Sattler et al., 2020) . On the other hand, it is reasonable to expect that users with simila

FAQs

What problem does FedCBO address in clustered federated learning?

FedCBO targets clustered federated learning where users belong to unknown groups and training should be effective for each group rather than only producing one global model.

How does the proposed method relate to consensus-based optimization?

The solution is inspired by consensus-based optimization and formulates the approach using an interacting-particle system that does not rely on knowing group memberships.

What theoretical results are provided for FedCBO?

The paper includes a mean-field analysis for the large number of particles limit and provides convergence guarantees for simultaneous global optimization of general non-convex objective functions in the mean-field regime.

Aug 3, 2026

Upload by Liam