[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-123224-en":3,"doc-seo-123224-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},123224,1099514067415,"Rowan","https://ap-avatar.wpscdn.com/avatar/100002539d78ffe74a7?x-image-process=image/resize,m_fixed,w_180,h_180&k=1779092875211072502",8,"Research & Report","PiPar - Pipeline parallelism for collaborative machine learning - Abstract and introduction","PiPar proposes a privacy-preserving approach for collaborative machine learning that improves low resource utilization observed in federated learning and related CML methods. The work identifies idle periods on both devices and servers caused primarily by sequential computation and communication. It introduces a training framework that uses pipeline parallelism to overlap computations across hardware resources and overlap communications across bandwidth resources. An automated, low-overhead parameter selection method optimizes the pipeline to maximize utilization. Experiments across multiple deep neural networks and datasets show large reductions in server idle time and significant training-time acceleration without sacrificing accuracy, including under differential privacy and heterogeneous device conditions.","PiPar: Pipeline parallelism for collaborative  \nmachine Learning  \nZihan Zhang, Philip Rodgers, Peter Kilpatrick, Ivor Spence, and Blesson Varghese  \narXiv :2302 . 12803v2 [ cs .DC] 25 Jun 2024  \nAbstract—Collaborative machine learning (CML) techniques, such as federated learning, have been proposed to train deep learning models across multiple mobile devices and a server. CML techniques are privacy-preserving as a local model that is trained on each device instead of the raw data from the device is shared with the server. However, CML training is inefficient due to low resource utilization. We identify idling resources on the server and devices due to sequential computation and communication asthe principal cause of low resource utilization. A novel framework PiPar that leverages pipeline parallelism for CML techniques is developed to substantially improve resource utilization. A new training pipeline is designed to parallelize the computations on different hardware resources and communication on different bandwidth resources, thereby accelerating the training process in CML. A low overhead automated parameter selection method is proposed to optimize the pipeline, maximizing the utilization of available resources. The experimental results confirm the validity of the underlying approach of PiPar and highlight that when compared to federated learning: (i) the idle time of the server can be reduced by up to 64.1×, and (ii) the overall training time can be accelerated by up to 34.6× under varying network conditions for a collection of six small and large popular deep neural networks and four datasets without sacrificing accuracy. It is also experimentally demonstrated that PiPar achieves performance benefits when incorporating differential privacy methods and operating in environments with heterogeneous devices and changing bandwidths.  \nIndex Terms—Collaborative machine learning, resource utilisation, pipeline parallelism, edge computing.  \nI. INTRODUCTION  \nDeep learning has found application across a range of fields including computer vision [1, 2], natural language processing [3, 4] and speech recognition [5, 6] . However, there are important data privacy and regulatory concerns in sending data generated on mobile devices to geographically distant cloud servers for training deep learning models. A new class of machine learning techniques has therefore been developed under the umbrella of collaborative machine learning (CML) to mitigate these concerns [7] . CML does not require data tobe sent to a server for training deep learning models; rather the server shares models with devices that are then locally trained on the device.  \nCML is used in many real-world use-cases comprising a central server and multiple homogeneous mobile devices. Smartphone manufacturers, for example, analyze user data to improve the performance of a specific smartphone model [8,  \nZ. Zhang and B. Varghese are with the School of Computer Science,  \nUniversity of St Andrews, UK. Corresponding author: [zz66@st-andrews.ac.uk](zz66@st-andrews.ac.uk)  \nP. Rodgers is with Rakuten Mobile, Inc., Japan.  \nP. Kilpatrick and I. Spence are with the School of Electronics, Electrical Engineering and Computer Science, Queen’s University Belfast, UK.  \n9] . For instance, CML can be employed to analyze the battery usage patterns of individual users on their phones to offer personalized plans for optimizing battery life. Similarly, CML can be used to analyze the typing habits of the users and then automatically complete and correct the typing of the users.  \nThere are three notable CML techniques reported in the literature, namely federated learning (FL) [10, 11, 12, 13], split learning (SL) [14, 15] and split federated learning (SFL) [16, 17] . However, these techniques under-utilize both compute and network resources, which results in training times that do not meet real-world requirements. The cause of resource underutilization and the resulting performance inefficien","cbCaie8ewHUvMyGP","https://ap.wps.com/l/cbCaie8ewHUvMyGP","pdf",5444649,1,20,"English","en",105,"# Abstract and introduction\n## Collaborative machine learning motivation\n## Existing techniques and their limitations\n## Resource under-utilization causes\n### Sequential execution\n### Communication bottlenecks","[{\"question\":\"What problem does PiPar address in collaborative machine learning?\",\"answer\":\"PiPar targets inefficient training caused by low resource utilization, where servers and devices remain idle due to sequential computation and communication.\"},{\"question\":\"How does PiPar improve efficiency compared with federated learning?\",\"answer\":\"PiPar uses pipeline parallelism to overlap computations on different hardware resources and communications on different bandwidth resources, reducing idle time and accelerating training.\"},{\"question\":\"What optimization does PiPar apply to the training pipeline?\",\"answer\":\"PiPar introduces a low-overhead automated parameter selection method to tune the pipeline for maximum utilization of available resources.\"}]","PiPar - Pipeline parallelism for collaborative machine learning - Abstract and introduction | PDF",1785815322,50,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"pipar-pipeline-parallelism-for-collaborative-machine-learning-abstract-and-introduction","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/pipar-pipeline-parallelism-for-collaborative-machine-learning-abstract-and-introduction/123224/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-04",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does PiPar address in collaborative machine learning?","Question",{"text":75,"@type":76},"PiPar targets inefficient training caused by low resource utilization, where servers and devices remain idle due to sequential computation and communication.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does PiPar improve efficiency compared with federated learning?",{"text":80,"@type":76},"PiPar uses pipeline parallelism to overlap computations on different hardware resources and communications on different bandwidth resources, reducing idle time and accelerating training.",{"name":82,"@type":73,"acceptedAnswer":83},"What optimization does PiPar apply to the training pipeline?",{"text":84,"@type":76},"PiPar introduces a low-overhead automated parameter selection method to tune the pipeline for maximum utilization of available resources.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,114,119,122,126,129,133],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":29,"slug":113},6,"Technology","technology",{"id":115,"doc_module":4,"doc_module_name":46,"category_name":116,"show_sort_weight":117,"slug":118},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":46,"category_name":124,"show_sort_weight":21,"slug":125},9,"Religion & Spirituality","religion-spirituality",{"id":21,"doc_module":4,"doc_module_name":46,"category_name":127,"show_sort_weight":21,"slug":128},"World Cup","world-cup",{"id":130,"doc_module":4,"doc_module_name":46,"category_name":131,"show_sort_weight":130,"slug":132},10,"Lifestyle","lifestyle",{"id":134,"doc_module":4,"doc_module_name":46,"category_name":135,"show_sort_weight":106,"slug":136},19,"General","general"]