[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-122847-en":3,"doc-seo-122847-105":30,"detail-sidebar-cat-0-en-105":83},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},122847,4810365810221,"Aurora","https://ap-avatar.wpscdn.com/davatar_155a257f0dc6eb9ab79c44ca47cae57d",8,"Research & Report","Efficient Resource Management for Machine Learning - Dissertation","Rising computational demands of machine learning and slower hardware progress create a compute supply-demand gap that raises training costs and limits availability for large models. The dissertation addresses this gap by increasing ML resource efficiency across multiple layers of the ML stack. It proposes Ekya, which improves continuous learning at the application layer via thief scheduling and micro-profiling; Cilantro, which uses online learning for performance-aware allocation in multi-tenant clusters; and ESCHER, which adds ephemeral resources to support custom scheduling needs without replacing the cluster manager.","UC Berkeley  \nUC Berkeley Electronic Theses and Dissertations  \nTitle  \nEfficient Resource Management for Machine Learning  \nPermalink  \n[https://escholarship.org/uc/item/3gd8d85s](https://escholarship.org/uc/item/3gd8d85s)  \nAuthor  \nBhardwaj, Romil  \nPublication Date  \n2023  \nPeer reviewed|Thesis/dissertation  \n[eScholarship.org](eScholarship.org) Powered by the California Digital Library  \nUniversity of California  \nEfficient Resource Management for Machine Learning  \nby Romil Bhardwaj  \nA dissertation submitted in partial satisfaction of the requirements for the degree of Doctor of Philosophy in Computer Science in the Graduate Division of the  \nUniversity of California, Berkeley  \nCommittee in charge:  \nProfessor Ion Stoica, Chair Dr. Ganesh Ananthanarayanan Professor Joseph E. Gonzalez Professor Scott Shenker  \nFall 2023  \nEfficient Resource Management for Machine Learning  \nCopyright 2023  \nby Romil Bhardwaj  \n1  \nAbstract  \nEfficient Resource Management for Machine Learning  \nby  \nRomil Bhardwaj  \nDoctor of Philosophy in Computer Science  \nUniversity of California, Berkeley  \nProfessor Ion Stoica, Chair  \nThe increasing computational demands of Machine Learning (ML) models, coupled with a slowdown in hardware advancements, have led to a significant compute supply-demand gap. This gap is evident in the rising costs and limited availability of resources needed for training complex ML models like GPT-4 . These challenges hinder the progress and accessibility of ML.  \nThis thesis aims to bridge the compute supply-demand gap by improving resource efficiency of ML. We introduce Ekya, Cilantro, and ESCHER, three new systems and methods for improving resource efficiency at different layer in the ML stack. Ekya, at the ML application layer, implementsa Thief Scheduling algorithm and a Microprofiler to intelligently redistribute resources between inference and retraining tasks, thereby making continuous learning four times more resourceefficient. Cilantro, in the cluster management layer, utilizes online learning to develop dynamic resource-performance models, enabling performance-aware resource allocation in multi-tenant environments. At the orchestration layer, ESCHER introduces ephemeral resources, allowing ML applications to specify custom scheduling requirements without overhauling the underlying cluster manager. This unique approach provides applications with the flexibility to adapt to evolving needs while maintaining simplicity in system design. Together, these systems represent a comprehensive approach to mitigating the compute supply-demand gap, contributing sustainable and efficient resource management techniques.  \ni  \nTo my parents.  \nii  \nContents  \nContents ii  \nList of Figures iv  \nList of Tables ix  \n1 Introduction 1  \n2 Ekya: Efficient Continuous Learning on the Edge 5  \n2.1 Introduction ...................................... 5  \n2.2 Continuous training on edge compute ........................ 8  \n2.2.1 Edge computing for video analytics ..................... 8  \n2.2.2 Compressed DNN models and data drift ................... 9  \n2.2.3 Accuracy benefits of continuous learning .................. 10  \n2.3 Scheduling retraining and inference jointly ...................... 11  \n2.3.1 Configuration diversity of retraining and inference ............. 11  \n2.3.2 Illustrative scheduling example ........................ 14  \n2.4 Ekya: Solution Description .............................. 15  \n2.4.1 Formulation of joint inference and retraining ................ 16  \n2.4.2 Thief Scheduler ................................ 16  \n2.4.3 Complexity analysis ............................. 19  \n2.4.4 Performance estimation with micro-profiling ................ 20  \n2.5 Implementation and Experimental Setup ....................... 21  \n2.6 Evaluation ....................................... 23  \n2.6.1 Overall improvements ............................ 23  \n2.6.2 Understanding Ekya’s improvements ..................... 27  \n2.6.3 Effectiv","cbCaioT5uwoyM89w","https://ap.wps.com/l/cbCaioT5uwoyM89w","pdf",5415106,1,124,"English","en",105,"# 1 Introduction\n# 2 Ekya: Efficient Continuous Learning on the Edge\n## 2.1 Introduction\n## 2.2 Continuous training on edge compute\n## 2.3 Scheduling retraining and inference jointly\n## 2.4 Ekya: Solution Description\n## 2.5 Implementation and Experimental Setup\n## 2.6 Evaluation\n## 2.7 Limitations and Discussion\n## 2.8 Related Work\n## 2.9 Conclusion\n# 3 Performance-aware Scheduling with Cilantro\n## 3.1 Introduction\n## 3.2 Background & Related Work\n## 3.3 Cilantro Architecture\n## 3.4 Policies\n## 3.5 Discussion\n## 3.6 Implementation\n## 3.7 Evaluation\n## 3.8 Conclusion\n# 4 ESCHER\n## 4.1 Introduction\n## 4.2 Motivation\n## 4.3 ESCHER Design and Workflow\n## 4.4 Scheduling with ESCHER","[{\"question\":\"How do Cilantro and ESCHER extend the approach across other layers?\",\"answer\":\"Cilantro improves cluster management by using online learning to build dynamic resource-performance models for performance-aware allocation in multi-tenant environments. ESCHER, at the orchestration layer, provides ephemeral resources so applications can specify custom scheduling requirements without redesigning the underlying cluster manager.\"}]","Efficient Resource Management for Machine Learning - Dissertation | PDF",1785813248,312,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":78,"head_meta":80,"extra_data":82,"updated_unix":28},"efficient-resource-management-for-machine-learning-dissertation","",{"@graph":36,"@context":77},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/efficient-resource-management-for-machine-learning-dissertation/122847/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-04",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71],{"name":72,"@type":73,"acceptedAnswer":74},"How do Cilantro and ESCHER extend the approach across other layers?","Question",{"text":75,"@type":76},"Cilantro improves cluster management by using online learning to build dynamic resource-performance models for performance-aware allocation in multi-tenant environments. ESCHER, at the orchestration layer, provides ephemeral resources so applications can specify custom scheduling requirements without redesigning the underlying cluster manager.","Answer","https://schema.org",{"og:url":52,"og:type":79,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":81,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":84},[85,89,93,97,102,107,112,115,120,123,127],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":86,"show_sort_weight":87,"slug":88},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":90,"show_sort_weight":91,"slug":92},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Exam",70,"exam",{"id":98,"doc_module":4,"doc_module_name":46,"category_name":99,"show_sort_weight":100,"slug":101},5,"Comic",60,"comic",{"id":103,"doc_module":4,"doc_module_name":46,"category_name":104,"show_sort_weight":105,"slug":106},6,"Technology",50,"technology",{"id":108,"doc_module":4,"doc_module_name":46,"category_name":109,"show_sort_weight":110,"slug":111},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":113,"slug":114},30,"research-report",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},9,"Religion & Spirituality",20,"religion-spirituality",{"id":118,"doc_module":4,"doc_module_name":46,"category_name":121,"show_sort_weight":118,"slug":122},"World Cup","world-cup",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":124,"slug":126},10,"Lifestyle","lifestyle",{"id":128,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":98,"slug":130},19,"General","general"]