[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-119383-en":3,"doc-seo-119383-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},119383,1099514067438,"River Wang","https://ap-avatar.wpscdn.com/avatar/100002539ee87300030?x-image-process=image/resize,m_fixed,w_180,h_180&k=1780474512215547542",8,"Research & Report","Neural Compression Inference Accelerator - A Cost and Energy-Effective Alternative to Conventional Machine Learning Inference Methods","Recent advances in machine learning and AI are driving demand for specialized cloud hardware, but GPUs and TPUs are costly to buy, expensive to rent, subject to pricing spikes, and highly energy intensive. The thesis investigates using field-programmable gate arrays (FPGAs) configured as machine learning accelerators. It proposes an FPGA architecture based on compressed neural networks and evaluates the design through architectural simulations and cloud deployment. Results show average 76.2% cost and 26.5% energy savings for neural inference workloads.","NEURAL COMPRESSION INFERENCE ACCELERATOR: A COST AND ENERGY-EFFECTIVE ALTERNATIVE TO CONVENTIONAL MACHINE LEARNING INFERENCE METHODS  \nA Thesis  \npresented to  \nthe Faculty of California Polytechnic State University, San Luis Obispo  \nIn Partial Fulfillment  \nof the Requirements for the Degree Master of Science in Electrical Engineering  \nby  \nMatthew Wallace  \n© 2023  \nMatthew Wallace  \nALL RIGHTS RESERVED  \nii  \nCOMMITTEE MEMBERSHIP  \nTITLE: Neural Compression Inference Accelerator: A Cost and Energy-Effective Alternative to Conventional Machine Learning Inference Methods  \nAUTHOR: Matthew Wallace  \nDATE SUBMITTED: June 2023  \nCOMMITTEE CHAIR: Joseph Callenes-Sloan, Ph.D.  \nProfessor of Computer Engineering  \nCOMMITTEE MEMBER: Andrew Danowitz, Ph.D.  \nProfessor of Computer Engineering  \nCOMMITTEE MEMBER: Helen Yu, Ph.D.  \nProfessor of Electrical Engineering  \nABSTRACT  \nNeural Compression Inference Accelerator: A Cost and Energy-Effective Alternative to Conventional Machine Learning Inference Methods  \nMatthew Wallace  \nRecent developments in machine learning and artificial intelligence have sparked an influx of workloads that require specialized computer hardware for cloud services. The hardware running machine learning models predominantly consists of graphics processing units (GPUs) and tensor processing units (TPUs) . However, these components are expensive for cloud services to purchase, costly for customers to rent, prone to price spikes, and energy-intensive. In this research we show that both cloud services and customers would benefit from utilizing field programmable gate arrays (FPGAs) to alleviate the aforementioned challenges. An FPGA can be configured asa machine learning accelerator, operating similarly to a GPU or TPU. We propose an FPGA-based architecture that utilizes compressed neural networks. This provides an alternative hardware option for those running machine learning inference, resulting in savings in server construction costs, power consumption, workload distribution, and providing a more affordable option for customers. We evaluate the system using architectural simulations and cloud-based deployment and demonstrate 76.2% cost and 26.5% energy savings on average from using the Neural Compression Inference Accelerator (NCIA) .  \nACKNOWLEDGMENTS  \nThanks to:  \n• Dr. Joseph Callenes-Sloan for advising me and guiding my research. I have learned so much about general computer engineering, modern technologies, how to implement novel systems, and how to effectively research directly from your classes and our meetings. I will have these new skills for life and am grateful to have learned them from someone so knowledgeable and experienced.  \n• Dr. Andrew Danowitz and Dr. Xiao-Hua Yu for reviewing and providing feedback for my work. A great fraction of my foundational knowledge and inspiration for my research comes from taking your EE509, EE521, and EE542 classes. Your captivating lectures helped me identify my field of passion in the large realm of engineering.  \n• My Mother, Father, and Sister for all of the love and support during my studies. I would not have been able to return to school and focus on learning without your support and guidance. I am forever grateful to have gone to graduate school to learn how to research and produce this Thesis. I am looking forward to celebrating and spending more time with you all soon.  \n• Emma for the care and calmness you bring to balance my work with life. It was always great knowing I could look forward to spending some relaxing time together after studying. Thank you for all the meals and favors you have done for me while I was glued to my computer.  \nTABLE OF CONTENTS  \nPage  \nLIST OF TABLES ................................. viii  \nLIST OF FIGURES ................................ ix  \nCHAPTER  \n1 Introduction ................................... 1  \n1.1 GPUs, FPGAs, the Cloud, and Machine Learning ........... 1  \n1.2 Contribution and Thesis Outline ...............","cbCainpN2IsPJp2N","https://ap.wps.com/l/cbCainpN2IsPJp2N","pdf",1729592,1,100,"English","en",105,"# Chapter 1 Introduction\n## 1.1 GPUs, FPGAs, the Cloud, and Machine Learning\n## 1.2 Contribution and Thesis Outline\n# Chapter 2 Compressing Neural Networks\n## 2.1 Pruning\n## 2.2 Quantization and Weight Sharing\n# Chapter 3 Neural Compression Inference Accelerator\n## 3.1 Hardware Parallelization\n## 3.2 Hardware Load Balancing\n# Chapter 4 Cloud Application: Amazon Web Services\n# Chapter 5 Methodology\n## 5.1 NCIA\n## 5.2 V100 GPU\n# Chapter 6 Results\n## 6.1 Performance\n## 6.2 Power Efficiency\n# Chapter 7 Discussion and Future Work\n## 7.1 Future of the NCIA\n## 7.2 Edge Applications\n# Chapter 8 Related Work\n## 8.1 Heterogeneous FPGAs in the Cloud\n## 8.2 Accelerating Machine Learning on the Edge\n# Chapter 9 Concl","[{\"question\":\"Why consider FPGA-based inference instead of conventional GPU/TPU approaches in the cloud?\",\"answer\":\"GPUs and TPUs are expensive to purchase, costly to rent, vulnerable to price spikes, and energy intensive. The thesis targets these cost and energy constraints for inference workloads.\"},{\"question\":\"What role does neural compression play in the proposed accelerator?\",\"answer\":\"The FPGA architecture uses compressed neural networks, enabling an alternative inference path that reduces server construction costs, power consumption, and improves workload affordability.\"},{\"question\":\"How are the accelerator benefits evaluated in the thesis?\",\"answer\":\"The work uses architectural simulations and cloud-based deployment to measure performance and quantify savings. The reported averages are 76.2% cost savings and 26.5% energy savings.\"}]","Neural Compression Inference Accelerator - A Cost and Energy-Effective Alternative to Conventional Machine Learning Inference Methods | PDF",1785724031,252,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"neural-compression-inference-accelerator-a-cost-and-energy-effective-alternative-to-conventional-machine-learning-inference-methods","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/neural-compression-inference-accelerator-a-cost-and-energy-effective-alternative-to-conventional-machine-learning-inference-methods/119383/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-03",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why consider FPGA-based inference instead of conventional GPU/TPU approaches in the cloud?","Question",{"text":75,"@type":76},"GPUs and TPUs are expensive to purchase, costly to rent, vulnerable to price spikes, and energy intensive. The thesis targets these cost and energy constraints for inference workloads.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What role does neural compression play in the proposed accelerator?",{"text":80,"@type":76},"The FPGA architecture uses compressed neural networks, enabling an alternative inference path that reduces server construction costs, power consumption, and improves workload affordability.",{"name":82,"@type":73,"acceptedAnswer":83},"How are the accelerator benefits evaluated in the thesis?",{"text":84,"@type":76},"The work uses architectural simulations and cloud-based deployment to measure performance and quantify savings. The reported averages are 76.2% cost savings and 26.5% energy savings.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]