[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-124804-en":3,"doc-seo-124804-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},124804,4810365810221,"Aurora","https://ap-avatar.wpscdn.com/davatar_155a257f0dc6eb9ab79c44ca47cae57d",8,"Research & Report","Learned Approximate Computing for Machine Learning","Deep neural networks drive rising computation demands, motivating approximate computing methods that trade accuracy for performance. This dissertation advances learned stochastic computing for neural networks, enabling adjustable precision with single-bit operations via probabilistic bit streams. It develops and improves multiple stochastic computing systems, beginning with 3pxnet for extreme quantization and pruning, then progressing through ACOUSTIC, GEO, and REX-SC to reduce accuracy gaps and optimize accumulation modeling and training pipelines. The work further generalizes learned training for other approximate-computing applications.","UCLA  \nUCLA Electronic Theses and Dissertations  \nTitle  \nLearned Approximate Computing for Machine Learning  \nPermalink  \n[https://escholarship.org/uc/item/1p7556nc](https://escholarship.org/uc/item/1p7556nc)  \nAuthor  \nLi, Tianmu  \nPublication Date  \n2023  \nPeer reviewed|Thesis/dissertation  \n[eScholarship.org](eScholarship.org) Powered by the California Digital Library  \nUniversity of California  \nUNIVERSITY OF CALIFORNIA Los Angeles  \nLearned Approximate Computing for Machine Learning  \nA dissertation submitted in partial satisfaction of the requirements for the degree Doctor of Philosophy in Electrical and Computer Engineering  \nby  \nTianmu Li  \n2023  \n© Copyright by Tianmu Li  \n2023  \nABSTRACT OF THE DISSERTATION  \nLearned Approximate Computing for Machine Learning  \nby  \nTianmu Li  \nDoctor of Philosophy in Electrical and Computer Engineering University of California, Los Angeles, 2023  \nProfessor Puneet Gupta, Chair  \nMachine learning using deep neural networks is growing in popularity and is demanding increasing computation requirements at the same time. Approximate computing is a promising approach that trades accuracy for performance, and stochastic computing is an especially interesting approach that preserves the compute units of single-bit computation while allowing adjustable compute precision. This dissertation centers around enabling and improving stochastic computing for neural networks, while also discussing works that lead up to stochastic computing and how the techniques developed for stochastic computing are applied to other approximate computing methods and applications other than deep neural networks. We start with 3pxnet, which combines extreme quantization with model pruning. While 3pxnet achieves extremely compact models, it demonstrates limits of binarization, includingthe inability to scale to higher precision levels and performance bottlenecks from accumulation. This leads us to stochastic computing, which performs single-gate multiplicationsand additions on probabilistic bit streams. The initial SC neural network implementation in ACOUSTIC aims at maximizing SC performance benefits while achieving usable accuracy. This is achieved through design choices in stream representation, performance optimizations  \nusing pooling layers, and training modifications to make single-gate accumulation possible. The subsequent work in GEO improves the stream generation and computation aspects of stochastic computing and reduces the accuracy gap between stochastic computing and fixed-point computing. The accumulation part of SC is further optimized in REX-SC, which allows efficient modeling of SC accumulation during training. During these iterations of the SC algorithm, we developed efficient training pipelines that target various aspects of training for approximate computing. Both forward and backward passes of training are optimized, which allows us to demonstrate model convergence results using SC and other approximate computing methods with limited hardware resources. Finally, we apply the training concept to other applications. In LAC, we show that an almost arbitrary parameterized application can be trained to perform well with approximate computing. At the same time, we can search for the optimal hardware configuration using NAS techniques.  \nThe dissertation of Tianmu Li is approved.  \nYizhou Sun  \nNader Sehatbakhsh  \nSudhakar Pamarti  \nPuneet Gupta, Committee Chair  \nUniversity of California, Los Angeles 2023  \nTo my parents for their support.  \nv  \nTABLE OF CONTENTS  \n1 Introduction ...................................... 1  \n1.1 Computation in deep neural networks ...................... 1  \n1.2 Computation with reduced accuracy ...................... 4  \n2 Sparse Binarized Processing ............................ 9  \n2.1 Introduction .................................... 9  \n2.1.1 A case for sparse XNOR networks .................... 10  \n2.2 Related Work ................................... 12  \n2.2.1 Binar","cbCaikQg3A42DFhr","https://ap.wps.com/l/cbCaikQg3A42DFhr","pdf",8042603,1,175,"English","en",105,"# Introduction\n## Computation in deep neural networks\n## Computation with reduced accuracy\n# Sparse Binarized Processing\n## Introduction\n## A case for sparse XNOR networks\n## Related Work\n### Binarized neural networks\n### Weight pruning\n### Sparse binary networks\n### Machine learning on embedded systems\n## The 3PXNet Approach\n## Experimental Setup\n## Results and Discussion\n## Conclusion\n# Making Stochastic Computing Work for Deep Learning\n## Stochastic Computing Primer\n## Stochastic Computing Baseline Implementation\n## Conclusion\n# Improving SC Generation and Execution\n## Co-optimized Shared Generation and Training\n## Partial Binary Accumulation","[{\"question\":\"Why is approximate computing important for deep neural networks?\",\"answer\":\"Deep neural network training and inference require increasing computation, and approximate computing improves performance by trading accuracy for efficiency. The dissertation positions stochastic computing as a promising path for adjustable precision.\"},{\"question\":\"What is the dissertation’s focus within stochastic computing?\",\"answer\":\"The work centers on enabling and improving stochastic computing for neural networks through stream representation choices, execution optimizations, and training modifications. It covers how techniques for stochastic computing extend to other approximate-computing methods and applications.\"},{\"question\":\"How does the dissertation evolve its approaches from 3pxnet to later systems?\",\"answer\":\"It begins with 3pxnet, combining extreme quantization and model pruning, and identifies limits in scaling to higher precision and accumulation bottlenecks. It then develops stochastic computing implementations (ACOUSTIC, GEO, REX-SC) to improve stream generation, reduce the accuracy gap with fixed-point computing, and efficiently model accumulation during training.\"}]","Learned Approximate Computing for Machine Learning | PDF",1785894744,441,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"learned-approximate-computing-for-machine-learning","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/learned-approximate-computing-for-machine-learning/124804/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-05",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why is approximate computing important for deep neural networks?","Question",{"text":75,"@type":76},"Deep neural network training and inference require increasing computation, and approximate computing improves performance by trading accuracy for efficiency. The dissertation positions stochastic computing as a promising path for adjustable precision.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What is the dissertation’s focus within stochastic computing?",{"text":80,"@type":76},"The work centers on enabling and improving stochastic computing for neural networks through stream representation choices, execution optimizations, and training modifications. It covers how techniques for stochastic computing extend to other approximate-computing methods and applications.",{"name":82,"@type":73,"acceptedAnswer":83},"How does the dissertation evolve its approaches from 3pxnet to later systems?",{"text":84,"@type":76},"It begins with 3pxnet, combining extreme quantization and model pruning, and identifies limits in scaling to higher precision and accumulation bottlenecks. It then develops stochastic computing implementations (ACOUSTIC, GEO, REX-SC) to improve stream generation, reduce the accuracy gap with fixed-point computing, and efficiently model accumulation during training.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]