[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-117782-en":3,"doc-seo-117782-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},117782,687197100911,"Himbo","https://ap-avatar.wpscdn.com/avatar/a000239b6f1da00475?x-image-process=image/resize,m_fixed,w_180,h_180&k=1785132997149421697",8,"Research & Report","Compositionality and Functorial Invariants in Machine Learning","This doctoral thesis demonstrates how analyzing the compositional and functorial structure underlying machine learning systems leads to a clearer understanding of their behavior. Category-theoretic formulations are developed across key subareas, including optimization, probability, unsupervised learning, and supervised learning. The thesis proves stability of optimization properties under relaxed assumptions and uses dynamical-system composition to build optimizers. It links maximum likelihood estimation to structure-preserving transformations, classifies unsupervised learning via functorial taxonomies, derives new clustering and manifold-learning algorithms, and formulates supervised learning problems as Kan extensions.","Compositionality and Functorial Invariantsin Machine Learning  \nDan Shiebler  \nKellogg College  \nUniversity of Oxford  \nA thesis submitted in fulfillment for the degree of Doctor of Philosophy  \nJanuary 26, 2023  \nAbstract  \nThe objective of this thesis is to show that studying the underlying compositional and functorial structure in machine learning systems allows us to better understand them. In order to do this, we explore category theoretic formulations of many subareas of machine learning, including optimization, probability, unsupervised learning, and supervised learning.  \nWe begin with an investigation of how various optimization algorithms behave when we replace the gradient with a generic category theoretic structure. We prove that the key properties of these algorithms hold under very relaxed assumptions, and demonstrate this result through numerical experiments. We also explore a category theoretic perspective on dynamical systems that enables us to build powerful optimizers from the composition of simple operations.  \nNext, we take a category theoretic perspective on the relationship between probabilistic modeling and gradient based optimization. We use this perspective to study how maximum likelihood estimation preserves certain key structures in the transformation from a statistical model to a supervised learning algorithm.  \nNext, we take a functorial perspective on unsupervised learning. We develop taxonomies of unsupervised learning algorithms based on the category theoretic properties of their functorial representations, and demonstrate that these taxonomies are predictive of algorithm behavior. We use this perspective to derive a host of new unsupervised learning algorithms for clustering and manifold learning, and demonstrate that these new algorithms can outperform commonly used alternatives on real world data. We also use these tools to prove new results on the behavior and limitations of popular unsupervised learning algorithms, including refinement bounds and stability in the face of noise.  \nFinally, we turn to supervised learning and demonstrate that many of the most common problems in data science and machine learning can be expressed as Kan extensions. We use this perspective to derive novel classification and supervised clustering algorithms. We also explore the performance of these algorithms on real data.  \nAcknowledgements  \nI am very grateful to my advisors Jeremy Gibbons and Cezar Ionescu whose thoughtful and wise guidance shaped the development of my research throughout the last several years.  \nI am also grateful to everyone who collaborated with me on my research, including Bruno Gavranović, Paul Wilson, Alexis Toumi, and Mehrnoosh Sadrzadeh. Our discussions helped my research reach new levels.  \nI also want to thank all of the awesome researchers who generously took the time to share feedback with me over the years. Leland McInnes, Luis Scoccola, and Jared Culbertson responded to my cold emails with compelling and thoughtful input. Noson Yanofsky has given me great feedback and a forum to present my research. Many anonymous conference and journal reviewers took the time to read my papers and share their thoughts. Thank you for your time and help.  \nI would also like to thank my DPhil examiners Sam Staton and Vitaliy Kurlin for volunteering their time and providing a careful examination.  \nI am grateful to Kellogg College, the Department of Continuing Education, the Department of Computer Science, and the wonderful staff for supporting me through the administrative aspects of this process.  \nI would also like to thank Twitter and the incredibly smart people who I have worked with over the last several years. I strongly believe that having one foot in academia and one in industry has made me both a stronger researcher and a stronger engineer.  \nI want to particularly thank my close friends and family who supported me as I navigated my DPhil journey. They have been patient with my focus bein","cbCaiv0Pr62kbFHt","https://ap.wps.com/l/cbCaiv0Pr62kbFHt","pdf",2622483,1,224,"English","en",105,"# Abstract\n# Optimization via category-theoretic structures\n# Category-theoretic view of probability and maximum likelihood\n# Functorial perspective on unsupervised learning\n# Supervised learning and Kan extensions\n# Acknowledgements\n# Statement of Originality\n# Publications","[{\"question\":\"What is the main objective of the thesis?\",\"answer\":\"To show that studying compositional and functorial structure in machine learning systems improves understanding of how these systems work.\"},{\"question\":\"How does the thesis approach optimization?\",\"answer\":\"It studies optimization algorithms when the gradient is replaced by a generic category-theoretic structure, proving key algorithm properties under relaxed assumptions and supporting results with numerical experiments.\"},{\"question\":\"What frameworks does the thesis use for unsupervised and supervised learning?\",\"answer\":\"For unsupervised learning, it builds taxonomies based on functorial representations’ category-theoretic properties. For supervised learning, it expresses many common problems as Kan extensions and derives related classification and supervised clustering algorithms.\"}]","Compositionality and Functorial Invariants in Machine Learning | PDF",1785679527,564,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"compositionality-and-functorial-invariants-in-machine-learning","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/compositionality-and-functorial-invariants-in-machine-learning/117782/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-02",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What is the main objective of the thesis?","Question",{"text":75,"@type":76},"To show that studying compositional and functorial structure in machine learning systems improves understanding of how these systems work.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does the thesis approach optimization?",{"text":80,"@type":76},"It studies optimization algorithms when the gradient is replaced by a generic category-theoretic structure, proving key algorithm properties under relaxed assumptions and supporting results with numerical experiments.",{"name":82,"@type":73,"acceptedAnswer":83},"What frameworks does the thesis use for unsupervised and supervised learning?",{"text":84,"@type":76},"For unsupervised learning, it builds taxonomies based on functorial representations’ category-theoretic properties. For supervised learning, it expresses many common problems as Kan extensions and derives related classification and supervised clustering algorithms.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]