[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-126589-en":3,"doc-seo-126589-105":31,"detail-sidebar-cat-0-en-105":92},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":28,"seo_description":14,"update_tm":29,"read_time":30},126589,687207017582,"Himbo","https://ap-avatar.wpscdn.com/davatar_994ba38a5ba835b3df7d355c54d3ed8d",8,"Research & Report","Improving Representation Learning through Variational Autoencoding - DPhil Thesis","Representation learning distills useful knowledge from raw data and enables generalization to new settings, forming a core step toward artificial intelligence. This DPhil thesis studies representations produced by variational autoencoders (VAEs), emphasizing their trainability, smooth latent spaces, and efficient compression into low-dimensional latent variables. It addresses two VAE training challenges—over-regularization and information drift—by optimizing decoder variance, using a flexible VAE-based prior, and adding a consistency loss to the ELBO objective.","University of Oxford  \nComputer Science Department DPhil Thesis  \nImproving Representation Learning through Variational Autoencoding  \nAuthor: Shuyu Lin, Balliol College  \nSupervisors: Professor Niki Trigoni  \nProfessor Stephen Roberts  \nAcknowledgements  \n“路漫漫其修远兮，吾将上下而求索”  \nTranslation: The path (to truth) is long and far, but I will not stop searching for it.  \n-Qu Yuan (c.340 BC – 278 BC), a Chinese poet and politician  \nI consider the five years in my PhD as a journey to pursuing truth-some very small pieces of unknown truth that keep me up at nights. As the Chinese poet wisely said 2 thousand years ago, this journey is not easy. The biggest challenges in this journey is doubt. I doubt myself. I doubt every decision I have made during my PhD: \"Am I working on an interesting topic?\" \"Am I asking the right question?\"\"Will this idea solve the problem?\" \"Ah it does not work very well, should I stick to it, or should I leave it and try something else?\" \"Ah something interesting seems happening, but what exactly is that?\" The second challenge is loneliness. Inspiring talks and engaging discussion with supervisors and colleagues can only help so far. Nobody shares the exact same objectives. Nobody wants to answer the exact same questions. To get to the truth I am after, I need to find my own way.  \nLuckily after five years of learning, asking questions, proposing ideas, submitting and re-submitting, I am here today-writing this last piece in my thesis. I want to thank many people - my parents, my supervisors, my friends, my cat and my boyfriend. It is their trust that I am able to complete this journey and to answer the questions that I want to ask that helps me to overcome doubt and loneliness. So I would like to dedicate this thesis to them and to this unforgettable journey of my life.  \nShuyu Lin, London, May 2022  \nAbstract  \nRepresentation learning aims to distill useful knowledge from raw data and apply this knowledge to a wide range of applications. This ability to extract information that is useful not only for selected tasks but also generalizes to new settings is a key step towards artificial intelligence.  \nIn this thesis, we focus on representations derived through a specific type of generative model, i.e. variational autoencoders (VAEs) . VAEs have several desirable properties. Thanks to the use of variational inference and the convenient model assumptions of Gaussian posteriors and a simple prior, VAEs are often easy to train and exhibit fast convergence. The probabilistic modelling formulation allows VAEs to derive a smooth latent representation of the raw data (i.e. semantically similar data samples are likely to be projected to nearby regions in the latent space) . VAEs compress the raw data to a much lower dimension latent space. Working with the low-dimensional representations rather than the raw data can significantly reduces costs in memory and computation. With these advantages, VAEs have been widely applied to many applications, including robotics [1], drug discovery [2] and digital content creation [3] .  \nDespite the widespread application of VAEs, improving the generative modelling of VAEs further remains an active research topic. In this thesis, we focus on two challenges in the VAE training: 1) over-regularized posterior distributions are often encountered in VAEs with Gaussian decoders and simple prior models; 2) the auto-encoding function may cause severe information drift and alter the information in the raw data in successive encodings. We propose solutions to both phenomena. Specifically, we optimize a variance parameter in the Gaussian decoder to balance competing loss terms in the ELBO objective. We adopt a flexible prior model that is implemented as a VAE in the latent space to mitigate the over-regularization effects. To reduce the information drift, we propose to modify the ELBO objective with a consistency loss that penalizes such drift. We show that these proposals can effectively address ","cbCaigKpPERe45qd","https://ap.wps.com/l/cbCaigKpPERe45qd","pdf",33678365,4,1,170,"English","en",105,"# Abstract\n## Representations and VAEs\n## Challenges and proposed solutions\n## Representation learning applications","[{\"question\":\"What is the main goal of representation learning in this thesis?\",\"answer\":\"Representation learning aims to extract useful knowledge from raw data and generalize it to new settings so it can support a wide range of applications.\"},{\"question\":\"Which two VAE training challenges does the thesis focus on?\",\"answer\":\"The thesis focuses on over-regularized posterior distributions and information drift caused by successive encodings during the auto-encoding process.\"},{\"question\":\"How does the thesis improve VAEs to address those training challenges?\",\"answer\":\"It optimizes a variance parameter in the Gaussian decoder to balance ELBO loss terms, introduces a flexible VAE-based prior to mitigate over-regularization, and adds a consistency loss to penalize information drift while improving likelihood.\"}]","Improving Representation Learning through Variational Autoencoding - DPhil Thesis | PDF",1785933527,428,{"code":4,"msg":32,"data":33},"ok",{"site_id":25,"language":24,"slug":34,"title":13,"keywords":35,"description":14,"schema_data":36,"social_meta":87,"head_meta":89,"extra_data":91,"updated_unix":29},"improving-representation-learning-through-variational-autoencoding-dphil-thesis","",{"@graph":37,"@context":86},[38,54,69],{"@type":39,"itemListElement":40},"BreadcrumbList",[41,45,49,52],{"item":42,"name":43,"@type":44,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":46,"name":47,"@type":44,"position":48},"https://docshare.wps.com/document/","Document",2,{"item":50,"name":12,"@type":44,"position":51},"https://docshare.wps.com/document/research-report/",3,{"item":53,"name":13,"@type":44,"position":20},"https://docshare.wps.com/document/improving-representation-learning-through-variational-autoencoding-dphil-thesis/126589/",{"url":53,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":24,"description":14,"dateModified":62,"datePublished":63,"encodingFormat":61,"isAccessibleForFree":64,"interactionStatistic":65},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":42,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-28","2026-08-05",true,{"@type":66,"interactionType":67,"userInteractionCount":20},"InteractionCounter",{"@type":68},"ViewAction",{"@type":70,"mainEntity":71},"FAQPage",[72,78,82],{"name":73,"@type":74,"acceptedAnswer":75},"What is the main goal of representation learning in this thesis?","Question",{"text":76,"@type":77},"Representation learning aims to extract useful knowledge from raw data and generalize it to new settings so it can support a wide range of applications.","Answer",{"name":79,"@type":74,"acceptedAnswer":80},"Which two VAE training challenges does the thesis focus on?",{"text":81,"@type":77},"The thesis focuses on over-regularized posterior distributions and information drift caused by successive encodings during the auto-encoding process.",{"name":83,"@type":74,"acceptedAnswer":84},"How does the thesis improve VAEs to address those training challenges?",{"text":85,"@type":77},"It optimizes a variance parameter in the Gaussian decoder to balance ELBO loss terms, introduces a flexible VAE-based prior to mitigate over-regularization, and adds a consistency loss to penalize information drift while improving likelihood.","https://schema.org",{"og:url":53,"og:type":88,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":90,"canonical":53},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":93},[94,98,102,106,111,116,121,124,129,132,136],{"id":21,"doc_module":4,"doc_module_name":47,"category_name":95,"show_sort_weight":96,"slug":97},"Story & Novel",90,"story-novel",{"id":48,"doc_module":4,"doc_module_name":47,"category_name":99,"show_sort_weight":100,"slug":101},"Literature",80,"literature",{"id":20,"doc_module":4,"doc_module_name":47,"category_name":103,"show_sort_weight":104,"slug":105},"Exam",70,"exam",{"id":107,"doc_module":4,"doc_module_name":47,"category_name":108,"show_sort_weight":109,"slug":110},5,"Comic",60,"comic",{"id":112,"doc_module":4,"doc_module_name":47,"category_name":113,"show_sort_weight":114,"slug":115},6,"Technology",50,"technology",{"id":117,"doc_module":4,"doc_module_name":47,"category_name":118,"show_sort_weight":119,"slug":120},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":47,"category_name":12,"show_sort_weight":122,"slug":123},30,"research-report",{"id":125,"doc_module":4,"doc_module_name":47,"category_name":126,"show_sort_weight":127,"slug":128},9,"Religion & Spirituality",20,"religion-spirituality",{"id":127,"doc_module":4,"doc_module_name":47,"category_name":130,"show_sort_weight":127,"slug":131},"World Cup","world-cup",{"id":133,"doc_module":4,"doc_module_name":47,"category_name":134,"show_sort_weight":133,"slug":135},10,"Lifestyle","lifestyle",{"id":137,"doc_module":4,"doc_module_name":47,"category_name":138,"show_sort_weight":107,"slug":139},19,"General","general"]