[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-127822-en":3,"doc-seo-127822-105":32,"detail-sidebar-cat-0-en-105":76},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":28,"seo_description":29,"update_tm":30,"read_time":31},127822,1099523882367,"Hazel","https://ap-avatar.wpscdn.com/davatar_9964176cb1d06d4a9deccf72a44ae3dc",8,"Research & Report","Software and hardware codesign of SmartNIC-based heterogeneous HPC clusters with machine learning case studies  ","\u003Cp>Machine learning has grown rapidly, driving larger models with massive memory footprints that outpace GPU memory capacity, creating the “memory wall” bottleneck for training. Heterogeneous training addresses this by combining CPUs, GPUs, and NVMe to offload parameters and states, yet performance remains constrained by data exchange, computation, and control efficiency. This dissertation proposes smartNIC-based heterogeneous systems and software-hardware codesign methods to improve scheduling, control, and communication through an intermediary caching layer and seamless connectivity between GPUs and offload engines.\u003C/p>","\u003Cp>Boston University  \u003C/p>\u003Cp>OpenBU [http://open. bu.edu](http://open. bu.edu)  \u003C/p>\u003Cp>Theses & Dissertations Boston University Theses & Dissertations  \u003C/p>\u003Cp>2024  \u003C/p>\u003Cp>Software and hardware codesign of SmartNIC-based heterogeneous HPC clusters with machine learning case studies  \u003C/p>\u003Cp>[https://hdl.handle.net/2144/49248](https://hdl.handle.net/2144/49248)[ ](https://hdl.handle.net/2144/49248)Boston University  \u003C/p>\u003Cp>BOSTON UNIVERSITY  \u003C/p>\u003Cp>COLLEGE OF ENGINEERING  \u003C/p>\u003Cp>Dissertation  \u003C/p>\u003Cp>SOFTWARE AND HARDWARE CODESIGN OF  \u003C/p>\u003Cp>SMARTNIC-BASED HETEROGENEOUS HPC  \u003C/p>\u003Cp>CLUSTERS WITH MACHINE LEARNING CASE  \u003C/p>\u003Cp>STUDIES  \u003C/p>\u003Cp>by  \u003C/p>\u003Cp>ANQI GUO  \u003C/p>\u003Cp>B.S., Lanzhou University, 2018  \u003C/p>\u003Cp>M.S., Boston University, 2020  \u003C/p>\u003Cp>Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy  \u003C/p>\u003Cp>© 2024 by ANQI GUO  \u003C/p>\u003Cp>All rights reserved  \u003C/p>\u003Cp>Approved by  \u003C/p>\u003Cp>First Reader  \u003C/p>\u003Cp>Martin C. Herbordt, PhD  \u003C/p>\u003Cp>Professor of Electrical and Computer Engineering  \u003C/p>\u003Cp>Second Reader  \u003C/p>\u003Cp>Roscoe Giles, PhD  \u003C/p>\u003Cp>Professor of Electrical and Computer Engineering  \u003C/p>\u003Cp>Third Reader  \u003C/p>\u003Cp>Tali Moreshet, PhD  \u003C/p>\u003Cp>Senior Lecturer and Research Assistant Professor of Electrical and Computer Engineering  \u003C/p>\u003Cp>Fourth Reader  \u003C/p>\u003Cp>Tong Geng, PhD  \u003C/p>\u003Cp>Assistant Professor of Electrical and Computer Engineering and Computer Science  \u003C/p>\u003Cp>University of Rochester  \u003C/p>\u003Cp>When pride comes, then comes disgrace, but with the humble is wisdom.  \u003C/p>\u003Cp>Proverbs 11:12  \u003C/p>\u003Cp>Acknowledgments  \u003C/p>\u003Cp>First and foremost, I would like to express my deepest gratitude to my advisor, Prof. Martin Herbordt. Throughout my PhD journey, his boundless help and guidance have been invaluable. His inspiration and detailed understanding of my project kept me motivated, and his consideration of my personal life deeply touched me. I am honored and proud to have had him as my advisor.  \u003C/p>\u003Cp>I would also like to extend my thanks to my other committee members, Professor Roscoe Giles, Professor Tali Moreshet, and Professor Tong Geng, for their invaluable feedback, generous support, and precious time. I am immensely grateful to all my collaborators and co-authors, especially Prof. Tong Geng, Ang Li, Yuchen Hao, Jianyu Huang, Cheng Tan, Cheng Yang, Qingqing Xiong, Pouya Haghi, and Chunshu Wu. Their contributions, expertise, and insights have significantly enriched the quality of my research.  \u003C/p>\u003Cp>Additionally, I would like to express my appreciation to all my friends and members of the CAAD lab for their support and encouragement. Special thanks to my friends Pouya, Chunshu, Sahan, Reza, Hafsah, and Zihao for the wonderful years shared during my PhD life.  \u003C/p>\u003Cp>Lastly, I want to thank my family for their unconditional and limitless support. Iam grateful to my dad and mom for their love and unwavering support of my decisions. I could not be who I am without my dearest family.  \u003C/p>\u003Cp>SOFTWARE AND HARDWARE CODESIGN OF SMARTNIC-BASED HETEROGENEOUS HPC CLUSTERS WITH MACHINE LEARNING CASE  \u003C/p>\u003Cp>STUDIES  \u003C/p>\u003Cp>ANQI GUO  \u003C/p>\u003Cp>Boston University, College of Engineering, 2024  \u003C/p>\u003Cp>Major Professor: Martin C. Herbordt, PhD  \u003C/p>\u003Cp>Professor of Electrical and Computer Engineering  \u003C/p>\u003Cp>ABSTRACT  \u003C/p>\u003Cp>Machine learning has evolved significantly recently and has penetrated every aspect of science, technology, and daily life. As application prediction demands higher accuracy and more complex tasks, larger models are proposed to meet these requirements. Deep learning applications like recommendation models and large language models have evolved with trillions of parameters and consume up to terabytes of memory. These models have outpaced the growth of GPU memories: GPU clusters, which aggregate GPU memory, have therefore grown exponentially to accommodate these large models. The Memory wall refers to the point at which the demand for memory exceeds the available capacity, creating a bottleneck for training ever-larger deep learning models. Heterogeneous deep learning training has become a key approach to addressing the limitations of GPU clusters, especially as models grow in size and complexity. By combining the strengths of CPUs, GPUs, and NVMe memory, heterogeneous systems aim to o\u003C/p>","cbCaidt8ypYwespG","https://ap.wps.com/l/cbCaidt8ypYwespG","pdf",13538915,6,1,245,"English","en",105,"# Abstract\n# Motivation and Background\n## Memory Wall and Heterogeneous Training\n## SmartNICs in Scale-Out Data Centers\n# Proposed SmartNIC-Based System Design\n## Heterogeneous System Design Steps\n## Software-Hardware Codesign for Performance\n## Dynamic Scheduling and Control\n## Caching and Reduced Communication Pressure\n# Machine Learning Case Studies","","Software and hardware codesign of SmartNIC-based heterogeneous HPC clusters with machine learning case studies   | PDF","Machine learning has grown rapidly, driving larger models with massive memory footprints that outpace GPU memory capacity, creating the “memory wall” bottleneck for training. Heterogeneous training addresses this by combining CPUs, GPUs, and NVMe to offload parameters and states, yet performance remains constrained by data exchange, computation, and control efficiency. This dissertation proposes smartNIC-based heterogeneous systems and software-hardware codesign methods to improve scheduling, control, and communication through an intermediary caching layer and seamless connectivity between GPUs and offload engines.",1787314052,617,{"code":4,"msg":33,"data":34},"ok",{"site_id":25,"language":24,"slug":35,"title":28,"keywords":27,"description":29,"schema_data":36,"social_meta":71,"head_meta":73,"extra_data":75,"updated_unix":30},"software-and-hardware-codesign-of-smartnic-based-heterogeneous-hpc-clusters-with-machine-learning-case-studies",{"@graph":37,"@context":70},[38,55],{"@type":39,"itemListElement":40},"BreadcrumbList",[41,45,49,52],{"item":42,"name":43,"@type":44,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":46,"name":47,"@type":44,"position":48},"https://docshare.wps.com/document/","Document",2,{"item":50,"name":12,"@type":44,"position":51},"https://docshare.wps.com/document/research-report/",3,{"item":53,"name":28,"@type":44,"position":54},"https://docshare.wps.com/document/software-and-hardware-codesign-of-smartnic-based-heterogeneous-hpc-clusters-with-machine-learning-case-studies/127822/",4,{"url":53,"name":28,"@type":56,"author":57,"headline":28,"publisher":59,"fileFormat":62,"inLanguage":24,"description":29,"dateModified":63,"datePublished":64,"encodingFormat":62,"isAccessibleForFree":65,"interactionStatistic":66},"DigitalDocument",{"name":9,"@type":58},"Person",{"url":42,"name":60,"@type":61},"DocShare","Organization","application/pdf","2026-09-03","2026-08-05",true,{"@type":67,"interactionType":68,"userInteractionCount":20},"InteractionCounter",{"@type":69},"ViewAction","https://schema.org",{"og:url":53,"og:type":72,"og:title":28,"og:site_name":60,"og:description":29},"article",{"robots":74,"canonical":53},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":77},[78,82,86,90,95,99,104,107,112,115,119],{"id":21,"doc_module":4,"doc_module_name":47,"category_name":79,"show_sort_weight":80,"slug":81},"Story & Novel",90,"story-novel",{"id":48,"doc_module":4,"doc_module_name":47,"category_name":83,"show_sort_weight":84,"slug":85},"Literature",80,"literature",{"id":54,"doc_module":4,"doc_module_name":47,"category_name":87,"show_sort_weight":88,"slug":89},"Exam",70,"exam",{"id":91,"doc_module":4,"doc_module_name":47,"category_name":92,"show_sort_weight":93,"slug":94},5,"Comic",60,"comic",{"id":20,"doc_module":4,"doc_module_name":47,"category_name":96,"show_sort_weight":97,"slug":98},"Technology",50,"technology",{"id":100,"doc_module":4,"doc_module_name":47,"category_name":101,"show_sort_weight":102,"slug":103},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":47,"category_name":12,"show_sort_weight":105,"slug":106},30,"research-report",{"id":108,"doc_module":4,"doc_module_name":47,"category_name":109,"show_sort_weight":110,"slug":111},9,"Religion & Spirituality",20,"religion-spirituality",{"id":110,"doc_module":4,"doc_module_name":47,"category_name":113,"show_sort_weight":110,"slug":114},"World Cup","world-cup",{"id":116,"doc_module":4,"doc_module_name":47,"category_name":117,"show_sort_weight":116,"slug":118},10,"Lifestyle","lifestyle",{"id":120,"doc_module":4,"doc_module_name":47,"category_name":121,"show_sort_weight":91,"slug":122},19,"General","general"]