[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-117158-en":3,"doc-seo-117158-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},117158,137441390410,"Hazel","https://ap-avatar.wpscdn.com/avatar/2000252f4ab5702993?_k=1776741390130283984",8,"Research & Report","Incorporating Inductive Biases into Machine Learning Algorithms","Recent advances in AI have enabled diffusion-based video generation from human descriptions and large language models to support writing, translation, and mathematical reasoning. These capabilities depend on training large deep-learning models on massive datasets, yet many important tasks—such as mathematical reasoning and molecule generation—face severe data scarcity. Even when models use almost all available Internet data, they remain imperfect. This thesis studies how inductive biases can improve AI performance by injecting human knowledge about data or tasks during structure design, training, and inference, supported by extensive experiments showing strong gains without additional data.","Incorporating Inductive Biases into Machine Learning Algorithms  \nNing Miao Wolfson College  \nUniversity of Oxford  \nA thesis submitted for the degree of Doctor of Philosophy in Statistics Trinity 2024  \nAcknowledgements  \nFirstly, I would love to express my sincerest gratitude to my supervisors Tom Rainforthand Yee Whye Teh for their kindness, patience, and invaluable guidance. I still remember when I just started my study in Oxford, Tom told me,‘You are not hereto help me, I’m here to help you.’ And this is exactly what he did. Over the past four years, we had countless discussions about my projects, from which I learned a lot. In particular, he would edit my drafts word by word, which took him a lot of time, but helped me a lot in improving my writing skills. I owe Yee Whye a huge debt of gratitude for his strong support, both academically and emotionally. Whenever my research reaches a deadlock, he is always able to give me hope and help me find a way forward. With the support from Tom and Yee Whye, I have grown from a shy, fresh DPhil student into a confident researcher who is proud of his work. They make me feel safe in academia, which allows me to do my research freely.  \nI would love to say ‘thank you’ to all my coauthors, including Emile Mathieu, N. Siddharth, Hyunjik Kim, Adam Foster, and Yann Dubois, as well as all members of RainML and Yee Whye group, especially Freddie Bickford Smith, Tim Reichelt, Andrew Campbell, Desi R. Ivanova, Jannik Kossen, and Jiazhan Feng. I would like to express a special thanks to Jin Xu for his help in every stage of my DPhil study. I would also like to thank my other friends in Oxford, especially Xi Lin, Yutong Lu, Chao Zhang, Zhixiao Zhu, Hanwen Xing, Yifan Yu, Linying Yang, Yanzhao Yang, Jun Yang, Zhongyi Hu, Jessie Jiang, and Alex Buna, as well as my old friends at home, especially Yuhao Jia, Dean Cai, Cheng Gao, Zhuofan Hao, Yu Gao, Chenxing Li, Shurun Wang, Hao Zhou, Wenxian Shi, and Hao Xu. Their companionship has made my life full of sunshine. I am also very grateful to my former supervisor Lei Li for his continuous support and guidance.  \nI appreciate the funding from Tencent, China Scholarship Council, and ELISE European Network of AI Excellence Centres, which allowed me to focus on my research without worrying about money.  \nI am deeply thankful to my family, especially my parents, for their unconditional love. Without their encouragement and financial support, it would have been impossible for me to even start my study at Oxford. Born in a small town in northern China, it is not easy to get the opportunity to pursue one’s academic dreams. I feel incredibly fortunate to be their child.  \nFinally, I would like to express my deepest gratitude to my partner Yao Li. She came to Oxford to accompany me since the start of my DPhil study. We have shared happiness and weathered difficult times together. She has always been there for me, and I cannot imagine the rest of my life without her.  \nAbstract  \nRecently, significant advances in artificial intelligence (AI) have surpassed what was imaginable even five years ago. Today, we can instruct diffusion-based models to generate high-quality videos from human descriptions or prompt large language models (LLMs) to assist with writing, translation, and even mathematical reasoning. These remarkable abilities arise from training massive deep-learning models on huge amounts of data. However, we do not always have enough data. In some tasks, such as mathematical reasoning or molecule generation, available data are very limited. Furthermore, despite current LLMs utilizing nearly all available data on the Internet, they remain imperfect. Thus, it is a critical question how to enhance the performance of AI systems when it is difficult to increase the amount of training data.  \nIn this thesis, we address this challenge from the perspective of inductive biases. Specifically, we investigate how to effectively use human knowledge about data or tasks ","cbCaicaj4n5VdVP8","https://ap.wps.com/l/cbCaicaj4n5VdVP8","pdf",9490879,1,146,"English","en",105,"# 1 Introduction\n## 1.1 Thesis outlines\n## 1.2 Included papers\n# 2 Literature Review and Discussion on Inductive Biases\n## 2.1 Definition and taxonomy of inductive biases\n## 2.2 Inductive biases of model structures\n## 2.3 Inductive biases of training algorithms\n## 2.4 Inductive biases of inference algorthms\n## 2.5 Connections with other machine learning concepts\n# 3 On Incorporating Inductive Biases into VAEs\n## 3.1 Introduction\n## 3.2 The need for inductive biases in VAEs\n## 3.3 Shortfalls of VAEs with non-Gaussian priors\n## 3.4 The InteL–VAE framework\n## 3.5 Related work","[{\"question\":\"Why is improving AI performance difficult when training data is limited?\",\"answer\":\"Many tasks such as mathematical reasoning and molecule generation have very limited data. Even large language models that use extensive Internet data remain imperfect, so performance must be improved without relying on larger datasets.\"},{\"question\":\"What does the thesis propose as an approach to handle data scarcity?\",\"answer\":\"The thesis addresses the challenge from the perspective of inductive biases, using human knowledge about data or tasks to optimize algorithm behavior without requiring extra data.\"},{\"question\":\"How are inductive biases incorporated according to the thesis?\",\"answer\":\"Inductive biases are introduced during structure design, training, and inference of machine learning models, with experiments showing performance improvements across multiple tasks.\"}]","Incorporating Inductive Biases into Machine Learning Algorithms | PDF",1785674155,368,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"incorporating-inductive-biases-into-machine-learning-algorithms","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/incorporating-inductive-biases-into-machine-learning-algorithms/117158/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-02",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why is improving AI performance difficult when training data is limited?","Question",{"text":75,"@type":76},"Many tasks such as mathematical reasoning and molecule generation have very limited data. Even large language models that use extensive Internet data remain imperfect, so performance must be improved without relying on larger datasets.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What does the thesis propose as an approach to handle data scarcity?",{"text":80,"@type":76},"The thesis addresses the challenge from the perspective of inductive biases, using human knowledge about data or tasks to optimize algorithm behavior without requiring extra data.",{"name":82,"@type":73,"acceptedAnswer":83},"How are inductive biases incorporated according to the thesis?",{"text":84,"@type":76},"Inductive biases are introduced during structure design, training, and inference of machine learning models, with experiments showing performance improvements across multiple tasks.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]