[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-123222-en":3,"doc-seo-123222-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},123222,1374391975076,"Riley","https://ap-avatar.wpscdn.com/avatar/14000253ca4ec9f6853?x-image-process=image/resize,m_fixed,w_180,h_180&k=1783305029341752051",8,"Research & Report","Lossy Compression with Machine Learning - Techniques and Fundamental Limits","This dissertation studies lossy compression by combining classical rate–distortion theory with neural compression methods, focusing on practical techniques and theoretical bounds. It develops novel inference optimization strategies for neural image compression, including hybrid amortized-iterative inference, stochastic Gumbel annealing, and lossy bits-back coding, and evaluates them through experiments. It further proposes asymmetrically-powered neural compression architectures to improve decoding efficiency and derives neural network-based sandwich bounds for the rate–distortion function. Finally, it presents statistical and optimal-transport perspectives for estimating the rate–distortion function, including Wasserstein gradient descent and sample-complexity analysis.","UC Irvine  \nUC Irvine Electronic Theses and Dissertations  \nTitle  \nLossy Compression with Machine Learning: Techniques and Fundamental Limits  \nPermalink  \n[https://escholarship.org/uc/item/1df6204v](https://escholarship.org/uc/item/1df6204v)  \nAuthor  \nYang, Yibo  \nPublication Date  \n2024  \nPeer reviewed|Thesis/dissertation  \n[eScholarship.org](eScholarship.org) Powered by the California Digital Library  \nUniversity of California  \nUNIVERSITY OF CALIFORNIA,  \nIRVINE  \nLossy Compression with Machine Learning:  \nTechniques and Fundamental Limits  \nDISSERTATION  \nsubmitted in partial satisfaction of the requirements  \nfor the degree of  \nDOCTOR OF PHILOSOPHY  \nin Computer Science  \nby  \nYibo Yang  \nDissertation Committee: Professor Stephan Mandt, Chair Professor Erik Sudderth Professor Weining Shen  \n© 2024 Yibo Yang  \nDEDICATION  \nTo my family.  \nTABLE OF CONTENTS  \nPage  \nLIST OF FIGURES vii  \nLIST OF TABLES x  \nACKNOWLEDGMENTS xi  \nVITA xiii  \nABSTRACT OF THE DISSERTATION xvi  \n1 Introduction 1  \n2 Background: Source Coding Fundamentals 9  \n2.1 The Overall Problem of Communication .................... 10  \n2.2 Lossless Compression ............................... 13  \n2.2.1 Optimal Symbol Code .......................... 14  \n2.2.2 Lossless Coding and Estimation ..................... 15  \n2.2.3 Huffman Coding ............................. 17  \n2.2.4 Arithmetic Coding ............................ 19  \n2.3 Lossy Compression ................................ 21  \n2.3.1 Rate-Distortion Theory .......................... 22  \n2.3.2 Vector Quantization ........................... 25  \n2.3.3 Transform Coding ............................. 27  \n3 Background: Neural Compression 29  \n3.1 Neural Lossless Compression ........................... 30  \n3.1.1 Continuous Models for Discrete Data .................. 31  \n3.1.2 Compression with Autoregressive Models ................ 33  \n3.1.3 Latent Variable Models and Bits-back Coding ............. 35  \n3.2 Neural Lossy Compression ............................ 41  \n3.2.1 Nonlinear Transform Coding ....................... 42  \n3.2.2 Channel Simulation and Relative Entropy Coding ........... 45  \n4 Inference Optimization 51  \n4.1 Introduction .................................... 52  \n4.2 Background: Lossy Neural Image Compression as Variational Inference ... 54  \n4.3 Novel Inference Techniques for Data Compression ............... 57  \n4.3.1 Amortization Gap and Hybrid Amortized-Iterative Inference ..... 58  \n4.3.2 Discretization Gap and Stochastic Gumbel Annealing (SGA) ..... 59  \n4.3.3 Marginalization Gap and Lossy Bits-Back Coding ........... 63  \n4.4 Experiments .................................... 66  \n4.5 Discussion ..................................... 69  \n5 Asymmetrically-Powered Neural Compression for Decoding Efficiency 71  \n5.1 Introduction .................................... 72  \n5.2 Background and Notation ............................ 74  \n5.2.1 Neural Image Compression ........................ 74  \n5.2.2 Iterative Inference ............................. 75  \n5.3 Methodology ................................... 76  \n5.3.1 The Case for a Shallow Decoder ..................... 76  \n5.3.2 Shallow Decoder Design ......................... 80  \n5.3.3 Formalizing the Role of the Encoder in Lossy Compression Performance 82  \n5.4 Experiments .................................... 84  \n5.4.1 Data and Training ............................ 84  \n5.4.2 Comparison with Existing Methods ................... 85  \n5.4.3 JPEG-Like Synthesis ........................... 88  \n5.4.4 Ablation Studies ............................. 89  \n5.5 Related Work ................................... 91  \n5.6 Discussion ..................................... 92  \n6 Neural Network-Based Sandwich Bounds on the Rate-Distortion Function 95  \n6.1 Introduction .................................... 96  \n6.2 Background .................................... 98  \n6.3 Upper Bound Algorithm ............................. 100  \n6.4 Lower Bound ","cbCaikVhFk4hxG1R","https://ap.wps.com/l/cbCaikVhFk4hxG1R","pdf",17275960,1,267,"English","en",105,"# 1 Introduction\n# 2 Background: Source Coding Fundamentals\n## 2.1 The Overall Problem of Communication\n## 2.2 Lossless Compression\n## 2.3 Lossy Compression\n# 3 Background: Neural Compression\n## 3.1 Neural Lossless Compression\n## 3.2 Neural Lossy Compression\n# 4 Inference Optimization\n## 4.2 Background: Lossy Neural Image Compression as Variational Inference\n## 4.3 Novel Inference Techniques for Data Compression\n# 5 Asymmetrically-Powered Neural Compression for Decoding Efficiency\n## 5.3 Methodology\n# 6 Neural Network-Based Sandwich Bounds on the Rate-Distortion Function\n## 6.3 Upper Bound Algorithm\n## 6.4 Lower Bound Algorithm\n# 7 Statistical and Optimal-Transport Perspectives on the Estimation of the Rate-Distortion Function\n## 7.2 Lossy Compression, Entropic Optimal Transport, and MLE\n## 7.4 Proposed Method\n# 8 Conclusion and Future Topics","[{\"question\":\"What are the main topics of this dissertation on lossy compression?\",\"answer\":\"It covers neural compression techniques integrated with rate–distortion theory, inference optimization for neural image compression, efficient decoding architectures, and theoretical estimation/bounds for the rate–distortion function.\"},{\"question\":\"How does the dissertation optimize inference for neural lossy compression?\",\"answer\":\"It introduces inference optimization methods including hybrid amortized-iterative inference, stochastic Gumbel annealing, and lossy bits-back coding, then validates the ideas via experiments and discussion.\"},{\"question\":\"What theoretical results are presented for the rate–distortion function?\",\"answer\":\"The work derives neural network-based sandwich bounds and develops an estimation framework using statistical and optimal-transport perspectives, including Wasserstein gradient descent and sample complexity analysis.\"}]","Lossy Compression with Machine Learning - Techniques and Fundamental Limits | PDF",1785815310,673,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"lossy-compression-with-machine-learning-techniques-and-fundamental-limits","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/lossy-compression-with-machine-learning-techniques-and-fundamental-limits/123222/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-04",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What are the main topics of this dissertation on lossy compression?","Question",{"text":75,"@type":76},"It covers neural compression techniques integrated with rate–distortion theory, inference optimization for neural image compression, efficient decoding architectures, and theoretical estimation/bounds for the rate–distortion function.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does the dissertation optimize inference for neural lossy compression?",{"text":80,"@type":76},"It introduces inference optimization methods including hybrid amortized-iterative inference, stochastic Gumbel annealing, and lossy bits-back coding, then validates the ideas via experiments and discussion.",{"name":82,"@type":73,"acceptedAnswer":83},"What theoretical results are presented for the rate–distortion function?",{"text":84,"@type":76},"The work derives neural network-based sandwich bounds and develops an estimation framework using statistical and optimal-transport perspectives, including Wasserstein gradient descent and sample complexity analysis.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]