[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-122897-en":3,"doc-seo-122897-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},122897,549758252649,"Ivy","https://ap-avatar.wpscdn.com/avatar/8000253669c5317157?_k=1778319167496531819",8,"Research & Report","PiPar: Pipeline parallelism for collaborative machine learning - Research paper","PiPar presents a pipeline-parallel framework for collaborative machine learning (CML) that addresses inefficiency caused by poor resource utilization in federated learning-style approaches. The work identifies server and device idling as the key issue stemming from sequential computation and communication patterns. PiPar builds a new training pipeline that parallelizes computation across different hardware resources and overlaps communication across bandwidth resources. It further introduces a low-overhead automated parameter selection method to maximize utilization. Experiments validate large reductions in server idle time and training acceleration without accuracy loss, and extend benefits to differential privacy and heterogeneous or changing-bandwidth environments.","Journal of Parallel and Distributed Computing 193 (2024) 104947  \nContents lists available at ScienceDirect  \nJournal of Parallel and Distributed Computing  \njournal [homepage: www.elsevier.com/locate/jpdc](homepage: www.elsevier.com/locate/jpdc)  \n| PiPar: Pipeline parallelism for collaborative machine learning |  |  |  |\n| --- | --- | --- | --- |\n| Zihan Zhang a,∗ , Philip Rodgers b, Peter Kilpatrick c, Ivor Spence c, Blesson Varghese a\u003Cbr>a School of Computer Science, University of St Andrews, St Andrews, United Kingdom b Rakuten Mobile Inc., Tokyo, Japan\u003Cbr>c School of Electronics, Electrical Engineering and Computer Science, Queen’s University Belfast, Belfast, United Kingdom |  |  |  |\n| A R T I C L E I N F O |  | A B S T R A C T |  |\n| Keywords:\u003Cbr>Collaborative machine learning Resource utilization\u003Cbr>Pipeline parallelism\u003Cbr>Edge computing |  | Collaborative machine learning (CML) techniques, such as federated learning, have been proposed to train deep learning models across multiple mobile devices and a server. CML techniques are privacy-preserving as a local model that is trained on each device instead of the raw data from the device is shared with the server. However, CML training is inefficient due to low resource utilization. We identify idling resources on the server and devices due to sequential computation and communication as the principal cause of low resource utilization. A novel framework PiPar that leverages pipeline parallelism for CML techniques is developed to substantially improve resource utilization. A new training pipeline is designed to parallelize the computations on different hardware resources and communication on different bandwidth resources, thereby accelerating the training process in CML. A low overhead automated parameter selection method is proposed to optimize the pipeline, maximizing the utilization of available resources. The experimental results confirm the validity of the underlying approach of PiPar and highlight that when compared to federated learning: (i) the idle time of the server can be reduced by up to 64.1×, and (ii) the overall training time can be accelerated by up to 34.6× under varying network conditions fora collection of six small and large popular deep neural networks and four datasets without sacrificing accuracy. It is also experimentally demonstrated that PiPar achieves performance benefits when incorporating differential privacy methods and operating in environments with heterogeneous devices and changing bandwidths. |  |\n\n1. Introduction  \nDeep learning has found application across a range of fields including computer vision [17,12], natural language processing [5,2] and speech recognition [7,10]. However, there are important data privacy and regulatory concerns in sending data generated on mobile devices to geographically distant cloud servers for training deep learning models. A new class of machine learning techniques has therefore been developed under the umbrella of collaborative machine learning (CML) to mitigate these concerns [38]. CML does not require data to be sent to a server for training deep learning models; rather the server shares models with devices that are then locally trained on the device.  \nCML is used in many real-world use-cases comprising a central server and multiple homogeneous mobile devices. Smartphone manufacturers, for example, analyse user data to improve the performance of a specific smartphone model [44,47]. For instance, CML can be employed to analyse the battery usage patterns of individual users on their phones tooffer personalized plans for optimizing battery life. Similarly, CML can  \n* Corresponding author.  \nE-mail address: [zz66@st-andrews.ac.uk](zz66@st-andrews.ac.uk) (Z. Zhang).  \nbe used to analyze the typing habits of the users and then automatically complete and correct the typing of the users.  \nThere are three notable CML techniques reported in the literature, namely federated learning (FL) [19–21,28], split learning (SL) [8,","cbCaifiM6aG7rJkX","https://ap.wps.com/l/cbCaifiM6aG7rJkX","pdf",6185727,1,17,"English","en",105,"# Introduction\n## Privacy concerns and collaborative machine learning (CML)\n## Common CML techniques: federated, split, and split federated learning\n## Root cause: resource under-utilization and idle time","[{\"question\":\"What problem does PiPar target in collaborative machine learning training?\",\"answer\":\"PiPar targets low resource utilization during CML training, caused by idling on both server and devices due to sequential computation and communication.\"},{\"question\":\"How does PiPar improve training efficiency?\",\"answer\":\"PiPar uses pipeline parallelism to overlap computation across hardware resources and communication across bandwidth resources, accelerating CML training.\"},{\"question\":\"What optimization does PiPar add to the pipeline?\",\"answer\":\"PiPar proposes a low-overhead automated parameter selection method to optimize the pipeline and maximize utilization of available resources.\"}]","PiPar: Pipeline parallelism for collaborative machine learning - Research paper | PDF",1785813555,43,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"pipar-pipeline-parallelism-for-collaborative-machine-learning-research-paper","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/pipar-pipeline-parallelism-for-collaborative-machine-learning-research-paper/122897/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-04",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does PiPar target in collaborative machine learning training?","Question",{"text":75,"@type":76},"PiPar targets low resource utilization during CML training, caused by idling on both server and devices due to sequential computation and communication.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does PiPar improve training efficiency?",{"text":80,"@type":76},"PiPar uses pipeline parallelism to overlap computation across hardware resources and communication across bandwidth resources, accelerating CML training.",{"name":82,"@type":73,"acceptedAnswer":83},"What optimization does PiPar add to the pipeline?",{"text":84,"@type":76},"PiPar proposes a low-overhead automated parameter selection method to optimize the pipeline and maximize utilization of available resources.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]