[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-120151-en":3,"doc-seo-120151-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},120151,8796095461610,"Oliver","https://ap-avatar.wpscdn.com/davatar_276721f389ce27ea32af1340a28f341c",8,"Research & Report","An Empirical Study of Decentralized Fine-tuning for Machine Learning Models","This thesis investigates decentralized machine learning when only limited data is available. Edge devices produce valuable data, yet central collection conflicts with data privacy regulations. Federated and gossip learning enable local training and iterative sharing to obtain an aggregated model, typically assuming ample data. Unlike prior work, this study focuses on decentralized fine-tuning under data scarcity, using extensive experiments across model architectures and datasets while varying distributions and imbalance to reflect realistic scenarios.","Universit􀀒a degli Studi di Padova  \nAn Empirical Study of Decentralized Fine-tuning for Machine Learning Models  \nDepartment: Mathematics  \nMajor: Computer Science  \nName: Yasser Nabil  \nSupervisor: Thomas Marchioro  \nAbstract  \nIn a world where data collected by edge devices is of high interest, gathering it at a central point has become more challenging due to current data protection regulations.  \nDecentralized paradigms, such as federated and gossip learning, have gained popularity as a solution to train models while avoiding to share raw data. In decentralized machine learning, edge devices train a local version of the model on their own private data. Moreover, local models are then iteratively shared and merged to produce an aggregated model. Federated learning has been proposed as an initial solution, which relies on a central server for the aggregation. In gossip learning, on the other hand, the model sharing and aggregation is performed in a peer-to-peer fashion, giving peers more control over which peers to collaborate with.  \nPrevious studies tested feasability and limits of decentralized machine learning training models from scratch, which requires access to a large pool of data. In this thesis, we analyze a di􀀋erent aspect of decentralized training, exploring the case of decentralized 􀀌ne-tuning of machine learning models with a limited amount of available data. Our research involves extensive experimentation with a number of model architectures and datasets, considering di􀀋erent data distributions and imbalances to simulate a real-life setup, demonstrating its applicability across di􀀋erent scenarios.  \nKeywords: Decentralized Machine Learning, Gossip Learning, Federated Learning, Fine-tuning, Model averaging.  \nPage 1 of 64  \nAcknowledgement  \nI want to express my gratitude to Dr. Thomas Marchioro, my thesis advisor, for always being there for me and pointing me in the correct direction whenever I needed it. Without his competent assistance and encouragement, my thesis would not have been doable. I would also like to thank Padua University for this opportunity given that helped me grow a lot and learn many important concepts and values.  \nPage 2 of 64  \nAbbreviations and Acronyms  \n\n| Abbreviation | Meaning |\n| --- | --- |\n| FL | Federated learning |\n| ML | Machine learning |\n| DML | Decentralized Machine learning |\n| AI | Arti􀀌cial Intelligence |\n| SGD | Stochastic Gradient Descent |\n| IoT | Internet of Things |\n| Non-IID | Non-Independent and Identically Distributed |\n| TN | True negative |\n| TP | True positive |\n| FN | False negative |\n| FP | False positive |\n\nPage 3 of 64  \nNotation Table  \n\n| Notation | Meaning |\n| --- | --- |\n| ¸ | The learning rate |\n| x | Feature vector |\n| y | Class label |\n| n | Number of training examples |\n| m | Number of target classes |\n| N | Number of peers |\n| w | Model parameters |\n| fw (x) | Model |\n| t | Number of iteration |\n| C | Set of classes |\n| N | Set of N peers |\n| D | Dataset |\n| B | Batch size |\n| J (w) | Objective function |\n| ℓ (fw (x), y) | Loss function |\n| L (x, y, w) | Per-example loss function |\n\nPage 4 of 64  \nContents  \n1 Introduction 7  \n2 Background 9  \n2.1 Machine Learning Fundamentals ................. 9  \n2.1.1 Goal of Machine Learning ................ 9  \n2.1.2 The Classi􀀌cation Task .................. 11  \n2.1.3 Generalization and Model Complexity ......... 14  \n2.1.4 Hyperparameters and Validation Sets .......... 15  \n2.1.5 Stochastic Gradient Descent ............... 16  \n2.2 Decentralized Machine Learning ................. 18  \n2.2.1 Federated Learning .................... 19  \n2.2.2 Gossip Learning ...................... 21  \n2.3 Fine-Tuning ............................ 23  \n2.4 Fundamentals of Network Science ................ 25  \n2.4.1 Degree Distribution .................... 25  \n2.4.2 Graph Representation and Adjacency Matrix ...... 26  \n2.4.3 Network models ...................... 27  \n3 Related Works 29  \n3.1 Non-IID Data ......................","cbCaikGygIVSajFb","https://ap.wps.com/l/cbCaikGygIVSajFb","pdf",1656873,1,65,"English","en",105,"# Contents\n## Introduction\n## Background\n## Related Works\n## Experimental Results and Analysis\n## Conclusion\n## Future Works","[{\"question\":\"Why is decentralized learning important in the thesis?\",\"answer\":\"Decentralized learning helps train models without sharing raw data, addressing challenges from data privacy regulations when data collection at a central point is restricted.\"},{\"question\":\"What is the thesis’s main focus regarding fine-tuning?\",\"answer\":\"The study analyzes decentralized fine-tuning of machine learning models when only a limited amount of data is available, rather than training from scratch with large datasets.\"},{\"question\":\"How does the thesis evaluate decentralized fine-tuning performance?\",\"answer\":\"Performance is assessed through extensive experiments using multiple model architectures and datasets, considering different data distributions and imbalance settings and comparing training strategies such as standalone, federated, and gossip learning.\"}]","An Empirical Study of Decentralized Fine-tuning for Machine Learning Models | PDF",1785728459,164,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"an-empirical-study-of-decentralized-fine-tuning-for-machine-learning-models","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/an-empirical-study-of-decentralized-fine-tuning-for-machine-learning-models/120151/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-03",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why is decentralized learning important in the thesis?","Question",{"text":75,"@type":76},"Decentralized learning helps train models without sharing raw data, addressing challenges from data privacy regulations when data collection at a central point is restricted.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What is the thesis’s main focus regarding fine-tuning?",{"text":80,"@type":76},"The study analyzes decentralized fine-tuning of machine learning models when only a limited amount of data is available, rather than training from scratch with large datasets.",{"name":82,"@type":73,"acceptedAnswer":83},"How does the thesis evaluate decentralized fine-tuning performance?",{"text":84,"@type":76},"Performance is assessed through extensive experiments using multiple model architectures and datasets, considering different data distributions and imbalance settings and comparing training strategies such as standalone, federated, and gossip learning.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]