[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-125859-en":3,"doc-seo-125859-105":31,"detail-sidebar-cat-0-en-105":93},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":28,"seo_description":14,"update_tm":29,"read_time":30},125859,1099523882367,"Hazel","https://ap-avatar.wpscdn.com/davatar_9964176cb1d06d4a9deccf72a44ae3dc",8,"Research & Report","On the Burstiness of Distributed Machine Learning Traffic","Enterprise data centers increasingly carry network traffic generated by distributed machine learning training, yet the bursty nature of this traffic has received limited study. Using measurements from a testbed network, the work analyzes short-term burstiness produced while training ResNet-50, introducing metrics to quantify burstiness across time scales. Results show strong burstiness at short intervals, with peak-to-mean ratios exceeding 60:1 up to 5 ms. The study links the burst pattern to training software coordination that avoids congestion and packet loss, and evaluates implications for congestion and flow control.","1  \narXiv :2401 .00329v1 [ cs .LG] 30 Dec 2023  \nOn the Burstiness of Distributed Machine Learning  \nTraffic  \nNatchanon Luangsomboon, Fahimeh Fazel, Jrg Liebeherr Ashkan Sobhani, Shichao Guan, Xingjun Chu  \nAbstract  \nTraffic from distributed training of machine learning (ML) models makes up a large and growing fraction of the traffic mix in enterprise data centers. While work on distributed ML abounds, the network traffic generated by distributed ML has received little attention. Using measurements on a testbed network, we investigate the traffic characteristics generated by the training of the ResNet-50 neural network with an emphasis on studying its shortterm burstiness. For the latter we propose metrics that quantify traffic burstiness at different time scales. Our analysis reveals that distributed ML traffic exhibits a very high degree of burstiness on short time scales, exceeding a 60:1 peak-to-mean ratio on time intervals as long as 5 ms. We observe that training software orchestrates transmissionsin such a way that burst transmissions from different sources within the same application do not result in congestion and packet losses. An extrapolation of the measurement data to multiple applications underscores the challenges of distributed ML traffic for congestion and flow control algorithms.  \nI. INTRODUCTION  \nThis paper studies and analyzes the burstiness of traffic from training deep neural network (DNN) models as a root cause for short-lived surges of traffic, known as microbursts, that cause periods of high packet delay and loss in a data center network (DCN) even at a low utilization. Since microbursts occur at a time scale of less than a millisecond [1], traditional traffic control methods are not effective with avoiding packet losses in such scenarios. Research on microbursts in DCNs has suggested a range of potential root causes, including the inherent burstiness of application traffic, confluence of traffic flows to a common destination (fan-in, incast), offloading of protocol processing at hosts, and traffic control algorithms, such as packet scheduling and flow control [1]–[10] . While training of neural networks makes up a large fraction of the workload in data centers [11], to the best of our knowledge, there does not exist a detailed analysis of distributed ML traffic and its potential impact on the creation of microbursts.  \nThe vast majority of network traffic from training DNN models is due to the exchange of gradients of model parameters. As modern DNN models involve millions, and, in the case of large language models such as GPT, billions of parameters [12], the transmission of gradients creates huge data bursts. In this paper, we present measurement experiments of the training of ResNet-50, a convolutional neural network for image classification with a total of 25 million parameters [13] . The measurement experiments are performed in a testbed network with a single switch with 100 Gbps line rates. We evaluate a server-based and a serverless mode of training. In server-based training, the nodes involved in the training, referred to as workers, exchange gradients with a dedicated server. Here, the transmissions to the server create a bottleneck. Serverless training avoids this pitfall by exchanging gradients in a distributed fashion. The measurement experiments in this paper use Ring Allreduce [14], a widely used technique for serverless gradient aggregation, where nodes are arranged in a logical ring, and gradients are disseminated along the ring.  \nN. Luangsomboon, F. Fazel, and J. Liebeherr are with the Department of Electrical and Computer Engineering, University of Toronto. A. Sobhani, S. Guan, and Xingjun Chu are with Huawei Canada, Ottawa.  \n2  \nA challenge for assessing the short-term burstiness of machine learning traffic is the lack of suitable metrics for quantifying and comparing burstiness properties. Metrics that are computed over an entire traffic trace, such as the ratio of the peak r","cbCaikRshGLzlV83","https://ap.wps.com/l/cbCaikRshGLzlV83","pdf",7202152,9,1,25,"English","en",105,"# Introduction\n## Motivation: Microbursts in data center networks\n## Gradient traffic in distributed training\n# Burstiness Metrics and Methodology\n## Why standard metrics fail\n## Network-calculus-based metrics\n# ResNet-50 Training Results\n## On-off traffic pattern\n## Peak-to-mean burstiness and coordination","[{\"question\":\"What is the main focus of this paper?\",\"answer\":\"The paper studies and analyzes how distributed machine learning training generates bursty network traffic, especially the short-term surge behavior linked to microbursts.\"},{\"question\":\"Why do traditional traffic metrics struggle with measuring burstiness here?\",\"answer\":\"Metrics computed over whole traces, such as peak-to-average ratios or percentiles, can miss the time scale of bursts and may fail to capture isolated large burst events.\"},{\"question\":\"What do the measurements on ResNet-50 show about burstiness?\",\"answer\":\"The traffic exhibits very high short-term burstiness, reaching peak-to-mean ratios above 60:1 for time intervals up to 5 ms, following an on-off pattern related to backpropagation.\"}]","On the Burstiness of Distributed Machine Learning Traffic | PDF",1785901629,63,{"code":4,"msg":32,"data":33},"ok",{"site_id":25,"language":24,"slug":34,"title":13,"keywords":35,"description":14,"schema_data":36,"social_meta":88,"head_meta":90,"extra_data":92,"updated_unix":29},"on-the-burstiness-of-distributed-machine-learning-traffic","",{"@graph":37,"@context":87},[38,55,70],{"@type":39,"itemListElement":40},"BreadcrumbList",[41,45,49,52],{"item":42,"name":43,"@type":44,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":46,"name":47,"@type":44,"position":48},"https://docshare.wps.com/document/","Document",2,{"item":50,"name":12,"@type":44,"position":51},"https://docshare.wps.com/document/research-report/",3,{"item":53,"name":13,"@type":44,"position":54},"https://docshare.wps.com/document/on-the-burstiness-of-distributed-machine-learning-traffic/125859/",4,{"url":53,"name":13,"@type":56,"author":57,"headline":13,"publisher":59,"fileFormat":62,"inLanguage":24,"description":14,"dateModified":63,"datePublished":64,"encodingFormat":62,"isAccessibleForFree":65,"interactionStatistic":66},"DigitalDocument",{"name":9,"@type":58},"Person",{"url":42,"name":60,"@type":61},"DocShare","Organization","application/pdf","2026-08-25","2026-08-05",true,{"@type":67,"interactionType":68,"userInteractionCount":20},"InteractionCounter",{"@type":69},"ViewAction",{"@type":71,"mainEntity":72},"FAQPage",[73,79,83],{"name":74,"@type":75,"acceptedAnswer":76},"What is the main focus of this paper?","Question",{"text":77,"@type":78},"The paper studies and analyzes how distributed machine learning training generates bursty network traffic, especially the short-term surge behavior linked to microbursts.","Answer",{"name":80,"@type":75,"acceptedAnswer":81},"Why do traditional traffic metrics struggle with measuring burstiness here?",{"text":82,"@type":78},"Metrics computed over whole traces, such as peak-to-average ratios or percentiles, can miss the time scale of bursts and may fail to capture isolated large burst events.",{"name":84,"@type":75,"acceptedAnswer":85},"What do the measurements on ResNet-50 show about burstiness?",{"text":86,"@type":78},"The traffic exhibits very high short-term burstiness, reaching peak-to-mean ratios above 60:1 for time intervals up to 5 ms, following an on-off pattern related to backpropagation.","https://schema.org",{"og:url":53,"og:type":89,"og:title":13,"og:site_name":60,"og:description":14},"article",{"robots":91,"canonical":53},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":94},[95,99,103,107,112,117,122,125,129,132,136],{"id":21,"doc_module":4,"doc_module_name":47,"category_name":96,"show_sort_weight":97,"slug":98},"Story & Novel",90,"story-novel",{"id":48,"doc_module":4,"doc_module_name":47,"category_name":100,"show_sort_weight":101,"slug":102},"Literature",80,"literature",{"id":54,"doc_module":4,"doc_module_name":47,"category_name":104,"show_sort_weight":105,"slug":106},"Exam",70,"exam",{"id":108,"doc_module":4,"doc_module_name":47,"category_name":109,"show_sort_weight":110,"slug":111},5,"Comic",60,"comic",{"id":113,"doc_module":4,"doc_module_name":47,"category_name":114,"show_sort_weight":115,"slug":116},6,"Technology",50,"technology",{"id":118,"doc_module":4,"doc_module_name":47,"category_name":119,"show_sort_weight":120,"slug":121},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":47,"category_name":12,"show_sort_weight":123,"slug":124},30,"research-report",{"id":20,"doc_module":4,"doc_module_name":47,"category_name":126,"show_sort_weight":127,"slug":128},"Religion & Spirituality",20,"religion-spirituality",{"id":127,"doc_module":4,"doc_module_name":47,"category_name":130,"show_sort_weight":127,"slug":131},"World Cup","world-cup",{"id":133,"doc_module":4,"doc_module_name":47,"category_name":134,"show_sort_weight":133,"slug":135},10,"Lifestyle","lifestyle",{"id":137,"doc_module":4,"doc_module_name":47,"category_name":138,"show_sort_weight":108,"slug":139},19,"General","general"]