[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-124876-en":3,"doc-seo-124876-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},124876,549758252649,"Ivy","https://ap-avatar.wpscdn.com/avatar/8000253669c5317157?_k=1778319167496531819",8,"Research & Report","Combining Cloud and Mobile Computing for Machine Learning - Abstract - Scheduler for model segmentation","Model sizes for machine learning continue to grow while mobile devices face strict constraints on memory capacity and battery life. Many current AI services execute inference entirely in the cloud, which motivates a more flexible, fine-grained split of computation. The approach segments neural models to offload compute-heavy layers to mobile or cloud while minimizing data transfer. A scheduler uses network quality, client device capability, and job requirements to keep consistent performance and reduce cloud work across heterogeneous devices.","Combining Cloud and Mobile Computing for Machine Learning  \nRuiqi Xu  \n[ruiqix@uchicago.edu](ruiqix@uchicago.edu)[ ](ruiqix@uchicago.edu)University of Chicago Chicago, Illinois, United States  \nTianchi Zhang  \n[tonyztc@uchicago.edu](tonyztc@uchicago.edu)[ ](tonyztc@uchicago.edu)University of Chicago Chicago, Illinois, United States  \narXiv :2402 .04880v2 [ cs .DC] 23 Feb 2024  \nABSTRACT  \nAlthough the computing power of mobile devices is increasing, machine learning models are also growing in size. This trend creates problems for mobile devices due to limitations like their memory capacity and battery life. While many services, like ChatGPT and Midjourney, run all the inferences in the cloud, we believe a flexible and fine-grained task distribution is more desirable. In this work, we consider model segmentation as a solution to improving the user experience, dividing the computation between mobile devices and the cloud in a way that offloads the compute-heavy portion of the model while minimizing the data transfer required. We show that the division not only reduces the wait time for users but can also be fine-tuned to optimize the workloads ofthe cloud. To achieve that, we design a scheduler that collects information about network quality, client device capability, and job requirements, making decisions to achieve consistent performance across a range of devices while reducing the work the cloud needs to perform.  \nCCS CONCEPTS  \n• Computing methodologies → Machine learning; • Networks → Cloud computing; Network measurement; • Computer systems organization → Real-time system architecture.  \nKEYWORDS  \nAccelerator scheduling, server-client coordination, machine learning performance  \n1 MOTIVATION  \nWhile the successes of services based on large machine learning models like ChatGPT, Stable Diffusion, Midjourney, and DALL-E draw attention to the power of AI, a constant trend in the field is to make the models bigger. A survey found that the language models have grown by seven orders of magnitude from 1950 to 2018, and from 2018 to 2022, they have increased by another five orders of magnitude. Although vision models grow at a constant rate, the size still expands by seven times in this period[39] . Despite the fact that this trend makes running those models on mobile devices seem impossible, we cannot ignore the advancement of mobile accelerators. The developers of the MLPerf mobile inference benchmark found a 2x throughput increase and 12x latency reduction on mobile devices in a period of 6 months[21] . While many companies choose to host their services entirely on the cloud, we believe this decision gives up many optimization opportunities. According to a recent work studying network latency using Google’s Stadia cloud-gaming service, the round-trip time (RTT) values are consistently lower than 25 ms[11] . At the same time, the 5G wireless network technology is aiming for end-to-end latency on the order of 1 ms[27] . This means users can exchange data with the cloud at little cost. Thus, we think  \nit is possible to distribute the workload in a more flexible fashion by sharing the results of layers in the middle of a model, essentially allowing the cloud to collaborate with mobile devices on inference tasks.  \n2 INTRODUCTION  \nThe idea behind our work is that by sharing intermediate results of layers in a neural network, we allow a different device to pick up the leftover work, effectively dividing the inference task. We aim to study the effectiveness of model segmentation using two types of tasks. The first one is image recognition. To achieve high accuracy on classification tasks like ImageNet, many models require a large number of parameters. These models are usually formed by a set of independent residual layer blocks, each taking in the output of its predecessor. This structure makes it easy to grab the output of one residual layer block and share it with another device. However, most of these models already run well on m","cbCainTKRbWl8SyT","https://ap.wps.com/l/cbCainTKRbWl8SyT","pdf",3870129,1,12,"English","en",105,"# Abstract\n## Motivation\n## Introduction","[{\"question\":\"Why is fully cloud-based inference challenging for mobile devices?\",\"answer\":\"Mobile devices have limited memory capacity and battery life, and larger ML models increase the cost of running inference entirely on-device.\"},{\"question\":\"What is model segmentation in this work?\",\"answer\":\"Model segmentation shares intermediate layer outputs so another device can continue the remaining inference computation.\"},{\"question\":\"How does the proposed scheduler make offloading decisions?\",\"answer\":\"It collects network quality, client device capability, and job requirements, then selects layer division to achieve consistent performance and reduce the cloud workload while meeting a latency target.\"}]","Combining Cloud and Mobile Computing for Machine Learning - Abstract - Scheduler for model segmentation | PDF",1785895164,30,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"combining-cloud-and-mobile-computing-for-machine-learning-abstract-scheduler-for-model-segmentation","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/combining-cloud-and-mobile-computing-for-machine-learning-abstract-scheduler-for-model-segmentation/124876/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-05",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why is fully cloud-based inference challenging for mobile devices?","Question",{"text":75,"@type":76},"Mobile devices have limited memory capacity and battery life, and larger ML models increase the cost of running inference entirely on-device.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What is model segmentation in this work?",{"text":80,"@type":76},"Model segmentation shares intermediate layer outputs so another device can continue the remaining inference computation.",{"name":82,"@type":73,"acceptedAnswer":83},"How does the proposed scheduler make offloading decisions?",{"text":84,"@type":76},"It collects network quality, client device capability, and job requirements, then selects layer division to achieve consistent performance and reduce the cloud workload while meeting a latency target.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":29,"slug":121},"research-report",{"id":123,"doc_module":4,"doc_module_name":46,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":46,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":46,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":46,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]