[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-128018-en":3,"doc-seo-128018-105":31,"detail-sidebar-cat-0-en-105":96},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":28,"seo_description":14,"update_tm":29,"read_time":30},128018,962084928904,"Asher","https://ap-avatar.wpscdn.com/davatar_155a257f0dc6eb9ab79c44ca47cae57d",8,"Research & Report","Accelerating Heterogeneous Federated Learning with Closed-form Classifiers - Paper summary","Federated Learning (FL) methods often experience degraded convergence and reduced accuracy in highly statistically heterogeneous (non-IID) client settings, where data imbalance and distribution shift cause client drift and biased local solutions. The work introduces Federated Recursive Ridge Regression (FED 3R), which uses a closed-form Ridge Regression classifier computed from pre-trained features, achieving immunity to statistical heterogeneity and invariance to client sampling order. The approach reduces communication and computation costs by up to two orders of magnitude and enables FED 3R parameters to initialize a softmax classifier for FL fine-tuning.","POLITECNICO DI TORINO Repository ISTITUZIONALE  \nAccelerating Heterogeneous Federated Learning with Closed-form Classifiers  \nOriginal  \nAccelerating Heterogeneous Federated Learning with Closed-form Classifiers / Fanì, Eros; Camoriano, Raffaello; Caputo, Barbara; Ciccone, Marco. -ELETTRONICO. -235:(2024), pp. 13029-13048. ( Forty-first International Conference on Machine Learning (ICML) Wien, Austria July 21-July 27, 2024) .  \nAvailability:  \nThis version is available at: 11583/2990261 since: 2025-02-26T17:55:23Z  \nPublisher:  \nML Research Press  \nPublished DOI:  \nTerms of use:  \nThis article is made available under terms and conditions as specified in the corresponding bibliographic description in the repository  \nPublisher copyright  \n(Article begins on next page)  \n21 February 2026  \nAccelerating Heterogeneous Federated Learning with Closed-form Classifiers  \nEros Fanì 1 Raffaello Camoriano 1 2 Barbara Caputo 1 3 Marco Ciccone 1  \nAbstract  \nFederated Learning (FL) methods often struggle in highly statistically heterogeneous settings.  \nIndeed, non-IID data distributions cause client drift and biased local solutions, particularly pronounced in the final classification layer, negatively impacting convergence speed and accuracy. To address this issue, we introduce Federated Recursive Ridge Regression (FED 3R) . Our method fitsa Ridge Regression classifier computed in closed form leveraging pre-trained features. FED3R is immune to statistical heterogeneity and is invariant to the sampling order of the clients. Therefore, it proves particularly effective in cross-device scenarios. Furthermore, it is fast and efficient in terms of communication and computation costs, requiring up to two orders of magnitude fewer resources than the competitors. Finally, we propose to leverage the FED 3R parameters as an initialization for a softmax classifier and subsequently finetune the model using any FL algorithm (FED3R with Fine-Tuning, FED 3R+FT) . Our findings also indicate that maintaining a fixed classifier aids in stabilizing the training and learning more discriminative features in cross-device settings. Official website: [https://fed-3r.github.io/](https://fed-3r.github.io/) .  \n1. Introduction  \nFederated Learning (FL) (McMahan et al., 2017) provides a practical framework for training machine learning models collaboratively across distributed clients while ensuring privacy. This decentralized approach involves multiple communication rounds between clients and a central server. During each round, clients leverage their private data to improve their local models. Then, they send the model updates to the server, which aggregates them and transmits the improved model to the next set of clients for further improvement.  \n1Department of Computing and Control Engineering, Polytechnic University of Turin, Italy 2Istituto Italiano di Tecnologia, Genoa, Italy 3CINI Consortium, Rome, Italy. Correspondence to: Eros Fanì \u003Ceros.fani@polito.it> .  \nProceedings of the 41 st International Conference on Machine Learning, Vienna, Austria. PMLR 235, 2024 . Copyright 2024 by the author(s) .  \nWhile appealing, limiting the optimization on the client side presents several challenges. In real-world scenarios, billions of clients might be involved (Kairouz et al., 2021), and data are often collected based on user preferences (Tan et al., 2022), availability (Gu et al., 2021), geographical location (Hsu et al., 2020 ; Fantauzzo et al., 2022), or personal habits (Fallah et al., 2020 ; Yang et al., 2018) . This leads to data distributions across clients with inherent statistical heterogeneity in the form of quantity skewness (Li et al., 2020b ; Wang et al., 2020 ; Hsu et al., 2020), label skewness (Karimireddy et al., 2020b ; Li et al., 2022 ; Caldarola et al., 2022 ; Fanì et al., 2023), or domain shift (Fantauzzo et al., 2022 ; Nguyen et al., 2022 ; Liu et al., 2021) .  \nAs a result, training models that generalize well across the global underlying data distr","cbCairY4Uh4m3w8g","https://ap.wps.com/l/cbCairY4Uh4m3w8g","pdf",8124649,3,1,21,"English","en",105,"# Abstract\n# Introduction\n## Federated Learning setup and communication rounds\n## Challenges from non-IID heterogeneity\n## Limitations of client-side optimization\n## Drift concentrated in classification heads","[{\"question\":\"What problem does FED 3R address in federated learning?\",\"answer\":\"It addresses slow convergence and accuracy loss caused by statistical heterogeneity in non-IID client data, which leads to client drift and biased local solutions, especially in the final classification layer.\"},{\"question\":\"How does FED 3R compute the classifier?\",\"answer\":\"It fits a Ridge Regression classifier in closed form using pre-trained features, avoiding sensitivity to client heterogeneity and sampling order.\"},{\"question\":\"How can FED 3R be combined with a softmax classifier?\",\"answer\":\"FED 3R parameters can initialize a softmax classifier, after which the model can be fine-tuned using any FL algorithm (FED 3R with Fine-Tuning, FED 3R+FT).\"},{\"question\":\"Why does keeping a fixed classifier help training?\",\"answer\":\"Maintaining a fixed classifier stabilizes training and helps the model learn more discriminative features in cross-device federated settings.\"}]","Accelerating Heterogeneous Federated Learning with Closed-form Classifiers - Paper summary | PDF",1785943960,53,{"code":4,"msg":32,"data":33},"ok",{"site_id":25,"language":24,"slug":34,"title":13,"keywords":35,"description":14,"schema_data":36,"social_meta":91,"head_meta":93,"extra_data":95,"updated_unix":29},"accelerating-heterogeneous-federated-learning-with-closed-form-classifiers-paper-summary","",{"@graph":37,"@context":90},[38,54,69],{"@type":39,"itemListElement":40},"BreadcrumbList",[41,45,49,51],{"item":42,"name":43,"@type":44,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":46,"name":47,"@type":44,"position":48},"https://docshare.wps.com/document/","Document",2,{"item":50,"name":12,"@type":44,"position":20},"https://docshare.wps.com/document/research-report/",{"item":52,"name":13,"@type":44,"position":53},"https://docshare.wps.com/document/accelerating-heterogeneous-federated-learning-with-closed-form-classifiers-paper-summary/128018/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":24,"description":14,"dateModified":62,"datePublished":63,"encodingFormat":61,"isAccessibleForFree":64,"interactionStatistic":65},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":42,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-27","2026-08-05",true,{"@type":66,"interactionType":67,"userInteractionCount":20},"InteractionCounter",{"@type":68},"ViewAction",{"@type":70,"mainEntity":71},"FAQPage",[72,78,82,86],{"name":73,"@type":74,"acceptedAnswer":75},"What problem does FED 3R address in federated learning?","Question",{"text":76,"@type":77},"It addresses slow convergence and accuracy loss caused by statistical heterogeneity in non-IID client data, which leads to client drift and biased local solutions, especially in the final classification layer.","Answer",{"name":79,"@type":74,"acceptedAnswer":80},"How does FED 3R compute the classifier?",{"text":81,"@type":77},"It fits a Ridge Regression classifier in closed form using pre-trained features, avoiding sensitivity to client heterogeneity and sampling order.",{"name":83,"@type":74,"acceptedAnswer":84},"How can FED 3R be combined with a softmax classifier?",{"text":85,"@type":77},"FED 3R parameters can initialize a softmax classifier, after which the model can be fine-tuned using any FL algorithm (FED 3R with Fine-Tuning, FED 3R+FT).",{"name":87,"@type":74,"acceptedAnswer":88},"Why does keeping a fixed classifier help training?",{"text":89,"@type":77},"Maintaining a fixed classifier stabilizes training and helps the model learn more discriminative features in cross-device federated settings.","https://schema.org",{"og:url":52,"og:type":92,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":94,"canonical":52},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":97},[98,102,106,110,115,120,125,128,133,136,140],{"id":21,"doc_module":4,"doc_module_name":47,"category_name":99,"show_sort_weight":100,"slug":101},"Story & Novel",90,"story-novel",{"id":48,"doc_module":4,"doc_module_name":47,"category_name":103,"show_sort_weight":104,"slug":105},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":47,"category_name":107,"show_sort_weight":108,"slug":109},"Exam",70,"exam",{"id":111,"doc_module":4,"doc_module_name":47,"category_name":112,"show_sort_weight":113,"slug":114},5,"Comic",60,"comic",{"id":116,"doc_module":4,"doc_module_name":47,"category_name":117,"show_sort_weight":118,"slug":119},6,"Technology",50,"technology",{"id":121,"doc_module":4,"doc_module_name":47,"category_name":122,"show_sort_weight":123,"slug":124},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":47,"category_name":12,"show_sort_weight":126,"slug":127},30,"research-report",{"id":129,"doc_module":4,"doc_module_name":47,"category_name":130,"show_sort_weight":131,"slug":132},9,"Religion & Spirituality",20,"religion-spirituality",{"id":131,"doc_module":4,"doc_module_name":47,"category_name":134,"show_sort_weight":131,"slug":135},"World Cup","world-cup",{"id":137,"doc_module":4,"doc_module_name":47,"category_name":138,"show_sort_weight":137,"slug":139},10,"Lifestyle","lifestyle",{"id":141,"doc_module":4,"doc_module_name":47,"category_name":142,"show_sort_weight":111,"slug":143},19,"General","general"]