[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-128074-en":3,"doc-seo-128074-105":31,"detail-sidebar-cat-0-en-105":92},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":28,"seo_description":14,"update_tm":29,"read_time":30},128074,5909887254083,"Miles","https://ap-avatar.wpscdn.com/davatar_276721f389ce27ea32af1340a28f341c",8,"Research & Report","AUTO-PARALLELIZATION OF MACHINE-LEARNING DATAFLOW GRAPHS FOR CPU MULTICORES - Thesis","Several methods aim to accelerate machine learning and deep-learning training and inference, but many graph- and operator-parallelism techniques require costly search-space optimizations. This thesis targets inference on CPUs, where batch size is often 1 and edge execution can make existing approaches costly or impractical. A critical-path-based linear clustering method exploits parallel paths in ML dataflow graphs, augmented with hyperclustering for small batch sizes greater than 1. The approach uses task parallelization via cloning and simplifies graphs with dead-code elimination, generating readable and executable PyTorch/Python code from ONNX models through a tool called Ramiel.","© 2023 SRINJOY DAS  \nAUTO-PARALLELIZATION OF MACHINE-LEARNING DATAFLOW GRAPHS FOR  \nCPU MULTICORES  \nBY  \nSRINJOY DAS  \nTHESIS  \nSubmitted in partial fulfillment of the requirements for the degree of Master of Science in Computer Science in the Graduate College of the  \nUniversity of Illinois Urbana-Champaign, 2023  \nUrbana, Illinois  \nAdviser:  \nProfessor Lawrence Rauchwerger  \nABSTRACT  \nSeveral methods exist today to accelerate Machine Learning(ML)/Deep-Learning(DL) model performance for training and inference. However, modern techniques that rely on various graph and operator parallelism methodologies rely on search space optimizations which are costly in terms of power and hardware usage. Especially in the case of inference, when the batch size is 1 and execution is on Central Processing Units (CPUs) or at the edge, current techniques can become costly, complicated or inapplicable. To ameliorate this, we present a Critical-Path-based Linear Clustering approach to exploit inherent parallel pathsin ML dataflow graphs. We augment this with a new hyperclustering mechanism for small batch sizes > 1 which may be typical in inference scenarios. Our task parallelization approach further optimizes the structure of graphs via cloning and simplifies them via deadcode elimination. Contrary to other work, we generate readable and executable parallel Pytorch+Python code from input ONNX models via a new tool that we have built called Ramiel which allows us to benefit from other downstream acceleration techniques like intraop parallelism and potentially pipeline parallelism. Our preliminary results on several ML graphs demonstrate up to 1 .9 × speedup over serial execution and outperform some of the current mechanisms in both compile and runtimes. Lastly, our methods are lightweight and fast enough so that they can be used effectively for Artificial Intelligence (AI) at the edge.  \nACKNOWLEDGMENTS  \nThis work is a result of everyone in my life that’s got me to this stage and heartfelt thankyous would not be sufficient to express my gratitude. To my father, where my interest in computer science and compilers stems from, without whose guiding words and intelligence, this thesis might have been a shell of itself. To my mother, whose undying love and moral support was necessary for me to keep moving forward. To my grandmothers, who want to see nothing but my success in life, and shield me from any difficulty. To my late grandfathers, who would be elated to know how far I’ve come and revel in my achievements, I know that they are watching me with joy.  \nTo my myriad friends who have been my companions and shaped my thoughts and dreams. The ones sitting back home in India, the ones who have accompanied me across oceans tobe on the same path as me and have seen the willpower required to even stand on your own feet, before going on to be successful. The ones in my locality, my school and undergrad. They are all here with me at this precipice.  \nTo the lessons I learnt in high school, that faith and toil cannot be replaced. And that hardships forge stronger steel. Every single heart that has been touched, every fragment of energy that has been lent to me, growing and expanding as it powers me on, creating a path of pure light towards eternal destinations. I owe it to them to keep moving forward, to carry their hopes and dreams within me.  \nI could not end without thanking the University of Illinois Urbana-Champaign. A college I dreamt of getting in during my application cycle, but was wholly convinced would not be what was destined for me. Stepping in to the University has been a life-altering experience in various ways, and I would not be what I was today without the college. A thanks to the Department of Computer Science for accepting me into their Master’s program and giving me the platform to complete a thesis. Finally, to my advisor, Dr. Lawrence Rauchwerger whose ideas and experiences on these topics could fill up libraries.  \nTABLE OF CONT","cbCaic7WV0Utk5Q6","https://ap.wps.com/l/cbCaic7WV0Utk5Q6","pdf",9567982,3,1,59,"English","en",105,"# CHAPTER 1 INTRODUCTION\n## 1.1 Introduction\n## 1.2 The Contribution of this Thesis\n## 1.3 Dissertation Organization\n# CHAPTER 2 BACKGROUND AND RELATED WORK\n## 2.1 Introduction\n## 2.2 ML Dataflow Graph Optimization\n# CHAPTER 3 OBSERVATIONS ON STRUCTURES OF ML DATAFLOW GRAPHS\n## 3.1 Introduction\n## 3.2 Potential Parallelism for some ML Dataflow Graphs\n## 3.3 Some Dataflow Graph Metrics\n# CHAPTER 4 PARALLELIZATION APPROACH VIA LINEAR CLUSTERING\n## 4.1 Introduction\n## 4.2 Graph Creation Pass\n## 4.3 Distance Pass\n## 4.4 Critical Path-Based Clustering Pass\n## 4.5 Cluster Merging Pass\n# CHAPTER 5 HYPERCLUSTERING: BEYOND LINEAR CLUSTERING\n## 5.1 Introduction\n## 5.2 Hyperclustering\n## 5.3 Hypercluster Switching\n# CHAPTER 6 FURTHER OPTIMIZATIONS\n## 6.1 Introduction\n## 6.2 Constant Propagation (CP), Folding and Dead-Code Elimination(DCE)\n## 6.3 Fusion\n## 6.4 Quantization\n## 6.5 Dataflow Graph Cloning\n# CHAPTER 7 IMPLEMENTATION DETAILS AND RESULTS\n## 7.1 Introduction\n## 7.2 Ramiel-a Tool for Clustering and Code Generation\n## 7.3 Ameliorating Possible Message Deadlock Scenarios\n## 7.4 Technical Results","[{\"question\":\"Why do existing graph and operator parallelism techniques become costly for inference on CPUs?\",\"answer\":\"Many methods depend on search-space optimizations that incur high power and hardware costs. In inference scenarios with batch size 1 on CPUs or at the edge, the techniques can become expensive, complex, or even inapplicable.\"},{\"question\":\"What is the core idea behind the thesis’ critical-path-based linear clustering method?\",\"answer\":\"The method exploits inherent parallel paths in ML dataflow graphs by clustering work using a critical-path perspective. It aims to generate effective task parallelism without relying on overly expensive search optimizations.\"},{\"question\":\"How does the thesis generate parallel executable code from ONNX inputs?\",\"answer\":\"It introduces Ramiel, a tool that produces readable and executable parallel PyTorch+Python code from input ONNX models. The generated graphs also incorporate optimizations such as cloning and dead-code elimination, and can benefit from downstream acceleration like intraop parallelism and potentially pipelining.\"}]","AUTO-PARALLELIZATION OF MACHINE-LEARNING DATAFLOW GRAPHS FOR CPU MULTICORES - Thesis | PDF",1785944671,149,{"code":4,"msg":32,"data":33},"ok",{"site_id":25,"language":24,"slug":34,"title":13,"keywords":35,"description":14,"schema_data":36,"social_meta":87,"head_meta":89,"extra_data":91,"updated_unix":29},"auto-parallelization-of-machine-learning-dataflow-graphs-for-cpu-multicores-thesis","",{"@graph":37,"@context":86},[38,54,69],{"@type":39,"itemListElement":40},"BreadcrumbList",[41,45,49,51],{"item":42,"name":43,"@type":44,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":46,"name":47,"@type":44,"position":48},"https://docshare.wps.com/document/","Document",2,{"item":50,"name":12,"@type":44,"position":20},"https://docshare.wps.com/document/research-report/",{"item":52,"name":13,"@type":44,"position":53},"https://docshare.wps.com/document/auto-parallelization-of-machine-learning-dataflow-graphs-for-cpu-multicores-thesis/128074/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":24,"description":14,"dateModified":62,"datePublished":63,"encodingFormat":61,"isAccessibleForFree":64,"interactionStatistic":65},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":42,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-29","2026-08-05",true,{"@type":66,"interactionType":67,"userInteractionCount":20},"InteractionCounter",{"@type":68},"ViewAction",{"@type":70,"mainEntity":71},"FAQPage",[72,78,82],{"name":73,"@type":74,"acceptedAnswer":75},"Why do existing graph and operator parallelism techniques become costly for inference on CPUs?","Question",{"text":76,"@type":77},"Many methods depend on search-space optimizations that incur high power and hardware costs. In inference scenarios with batch size 1 on CPUs or at the edge, the techniques can become expensive, complex, or even inapplicable.","Answer",{"name":79,"@type":74,"acceptedAnswer":80},"What is the core idea behind the thesis’ critical-path-based linear clustering method?",{"text":81,"@type":77},"The method exploits inherent parallel paths in ML dataflow graphs by clustering work using a critical-path perspective. It aims to generate effective task parallelism without relying on overly expensive search optimizations.",{"name":83,"@type":74,"acceptedAnswer":84},"How does the thesis generate parallel executable code from ONNX inputs?",{"text":85,"@type":77},"It introduces Ramiel, a tool that produces readable and executable parallel PyTorch+Python code from input ONNX models. The generated graphs also incorporate optimizations such as cloning and dead-code elimination, and can benefit from downstream acceleration like intraop parallelism and potentially pipelining.","https://schema.org",{"og:url":52,"og:type":88,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":90,"canonical":52},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":93},[94,98,102,106,111,116,121,124,129,132,136],{"id":21,"doc_module":4,"doc_module_name":47,"category_name":95,"show_sort_weight":96,"slug":97},"Story & Novel",90,"story-novel",{"id":48,"doc_module":4,"doc_module_name":47,"category_name":99,"show_sort_weight":100,"slug":101},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":47,"category_name":103,"show_sort_weight":104,"slug":105},"Exam",70,"exam",{"id":107,"doc_module":4,"doc_module_name":47,"category_name":108,"show_sort_weight":109,"slug":110},5,"Comic",60,"comic",{"id":112,"doc_module":4,"doc_module_name":47,"category_name":113,"show_sort_weight":114,"slug":115},6,"Technology",50,"technology",{"id":117,"doc_module":4,"doc_module_name":47,"category_name":118,"show_sort_weight":119,"slug":120},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":47,"category_name":12,"show_sort_weight":122,"slug":123},30,"research-report",{"id":125,"doc_module":4,"doc_module_name":47,"category_name":126,"show_sort_weight":127,"slug":128},9,"Religion & Spirituality",20,"religion-spirituality",{"id":127,"doc_module":4,"doc_module_name":47,"category_name":130,"show_sort_weight":127,"slug":131},"World Cup","world-cup",{"id":133,"doc_module":4,"doc_module_name":47,"category_name":134,"show_sort_weight":133,"slug":135},10,"Lifestyle","lifestyle",{"id":137,"doc_module":4,"doc_module_name":47,"category_name":138,"show_sort_weight":107,"slug":139},19,"General","general"]