Software and hardware codesign of SmartNIC-based heterogeneous HPC clusters with machine learning case studies

Aug 21, 2026

Upload by Hazel

Machine learning has grown rapidly, driving larger models with massive memory footprints that outpace GPU memory capacity, creating the “memory wall” bottleneck for training. Heterogeneous training addresses this by combining CPUs, GPUs, and NVMe to offload parameters and states, yet performance remains constrained by data exchange, computation, and control efficiency. This dissertation proposes smartNIC-based heterogeneous systems and software-hardware codesign methods to improve scheduling, control, and communication through an intermediary caching layer and seamless connectivity between GPUs and offload engines.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages245

Reading Time10 h 17 min

AccessFree to download

Abstract

Motivation and Background

Memory Wall and Heterogeneous Training

SmartNICs in Scale-Out Data Centers

Proposed SmartNIC-Based System Design

Heterogeneous System Design Steps

Software-Hardware Codesign for Performance

Dynamic Scheduling and Control

Caching and Reduced Communication Pressure

Machine Learning Case Studies

Boston University

OpenBU [http://open. bu.edu](http://open. bu.edu)

Theses & Dissertations Boston University Theses & Dissertations

2024

Software and hardware codesign of SmartNIC-based heterogeneous HPC clusters with machine learning case studies

[https://hdl.handle.net/2144/49248](https://hdl.handle.net/2144/49248)[ ](https://hdl.handle.net/2144/49248)Boston University

BOSTON UNIVERSITY

COLLEGE OF ENGINEERING

Dissertation

SOFTWARE AND HARDWARE CODESIGN OF

SMARTNIC-BASED HETEROGENEOUS HPC

CLUSTERS WITH MACHINE LEARNING CASE

STUDIES

ANQI GUO

B.S., Lanzhou University, 2018

M.S., Boston University, 2020

Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy

Approved by

First Reader

Martin C. Herbordt, PhD

Professor of Electrical and Computer Engineering

Second Reader

Roscoe Giles, PhD

Professor of Electrical and Computer Engineering

Third Reader

Tali Moreshet, PhD

Senior Lecturer and Research Assistant Professor of Electrical and Computer Engineering

Fourth Reader

Tong Geng, PhD

Assistant Professor of Electrical and Computer Engineering and Computer Science

University of Rochester

When pride comes, then comes disgrace, but with the humble is wisdom.

Proverbs 11:12

Acknowledgments

First and foremost, I would like to express my deepest gratitude to my advisor, Prof. Martin Herbordt. Throughout my PhD journey, his boundless help and guidance have been invaluable. His inspiration and detailed understanding of my project kept me motivated, and his consideration of my personal life deeply touched me. I am honored and proud to have had him as my advisor.

I would also like to extend my thanks to my other committee members, Professor Roscoe Giles, Professor Tali Moreshet, and Professor Tong Geng, for their invaluable feedback, generous support, and precious time. I am immensely grateful to all my collaborators and co-authors, especially Prof. Tong Geng, Ang Li, Yuchen Hao, Jianyu Huang, Cheng Tan, Cheng Yang, Qingqing Xiong, Pouya Haghi, and Chunshu Wu. Their contributions, expertise, and insights have significantly enriched the quality of my research.

Additionally, I would like to express my appreciation to all my friends and members of the CAAD lab for their support and encouragement. Special thanks to my friends Pouya, Chunshu, Sahan, Reza, Hafsah, and Zihao for the wonderful years shared during my PhD life.

Lastly, I want to thank my family for their unconditional and limitless support. Iam grateful to my dad and mom for their love and unwavering support of my decisions. I could not be who I am without my dearest family.

SOFTWARE AND HARDWARE CODESIGN OF SMARTNIC-BASED HETEROGENEOUS HPC CLUSTERS WITH MACHINE LEARNING CASE

STUDIES

ANQI GUO

Boston University, College of Engineering, 2024

Major Professor: Martin C. Herbordt, PhD

Professor of Electrical and Computer Engineering

ABSTRACT

Machine learning has evolved significantly recently and has penetrated every aspect of science, technology, and daily life. As application prediction demands higher accuracy and more complex tasks, larger models are proposed to meet these requirements. Deep learning applications like recommendation models and large language models have evolved with trillions of parameters and consume up to terabytes of memory. These models have outpaced the growth of GPU memories: GPU clusters, which aggregate GPU memory, have therefore grown exponentially to accommodate these large models. The Memory wall refers to the point at which the demand for memory exceeds the available capacity, creating a bottleneck for training ever-larger deep learning models. Heterogeneous deep learning training has become a key approach to addressing the limitations of GPU clusters, especially as models grow in size and complexity. By combining the strengths of CPUs, GPUs, and NVMe memory, heterogeneous systems aim to o

Aug 21, 2026

Upload by Hazel