[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-118227-en":3,"doc-seo-118227-105":30,"detail-sidebar-cat-0-en-105":92},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},118227,8796095461610,"Oliver","https://ap-avatar.wpscdn.com/davatar_276721f389ce27ea32af1340a28f341c",8,"Research & Report","Sparse Parameterisations For Efficient Machine Learning Algorithms","Progress in machine learning increasingly depends on large models, massive datasets, and high-performance clusters, but the cost of distributed training limits participation. Even with strong hardware, inference often runs on constrained devices, requiring smaller models without sacrificing learned capabilities. Real-world settings further demand continual learning as data arrives sequentially. This thesis develops methods that improve efficiency through sparse parameterisations and task-based formalisations, enabling tractable procedures for data selection, model-size reduction, and high-plasticity parameter learning for rapid meta-learning adaptation. It highlights efficient data compression as a high-impact application area.","Sparse Parameterisations For Efficient Machine Learning Algorithms  \nJonathan Richard Schwarz  \nA dissertation submitted in partial fulfilment of the requirements for the degree of  \nDoctor of Philosophy  \nof  \nUniversity College London.  \nCoMPLEX  \nDepartment of Computer Science University College London  \nNovember 4, 2023  \nDeclaration  \nI, Jonathan Richard Schwarz, confirm that the work presented in this thesis is my own . Where information has been derived from other sources, I confirm that this  \nhas been indicated in the work.  \nAbstract  \nProgress in Machine Learning is being driven by continued growth in model size, training data and algorithmic innovations relying on access to high-performance computing clusters. While this paradigm has dominated with the advent of modern Deep Representation Learning, concerns over practical limitations are becoming increasingly common: Independent and identically distributed training on large datasets is prohibitively expensive for all but a handful of institutions, reducing participation and the pace of innovation . Even in cases where advanced training hardware is readily available, inference must often be carried out on limited hardware, posing the challenge of reducing model capacity once the functional relationship of a learning problem has been extracted. In addition, this paradigm conflicts with the inherent nature of real applications, where data is collected sequentially and thus Continual Learning is required. Hence, focus on efficiency must not merely be a burden or constraint but can instead both incentivise and benefit from knowledge transfer, leading to better generalisation. Finally, in an age of increasing concern about the environmental footprint of technology, reductions in computational requirements are not merely a cost-saving endeavour but critical to the long-term progress of the field.  \nIn this thesis, we develop algorithmic approaches towards increasing the efficiency of Machine Learning by utilising sparse parameterisations and formalising our techniques as operating on the explicit or tacit notion of several tasks. These principles will allow us to devise tractable computational procedures ranging from identifying the most informative subsets of data over drastic reductions in model size without performance loss to finding parameters with the highest plasticity,  \nallowing the rapid adaptation to a task through Meta Learning.  \nFinally, while covering various practical problems throughout the thesis, we will emphasise applications to data compression, a high-impact problem uniquely encapsulating both requirements and promises of Efficient Machine Learning.  \nContents  \nImpact Statement  \nThe impact of this thesis takes the form of primarily scholarly contributions likely to inspire future research but also has a direct path towards applications in both commercial settings or as a technology in scientific research. In addition, our unifying perspective encapsulates a range of works outside this thesis’s content. Hence, the explicit argument for this nascent development will likely inspire future work across a range of topics in Machine Learning.  \nSpecifically, our work on Functional Regularisation for Continual Learning (FRCL) led to several closely related or directly inspired results (Pan et al. , 2020 ; Verma et al. , 2022 ; Rudner et al. , 2022) through its combination of previously distinct approaches to CL and the first use of Gaussian Processes to optimise rehearsal data. This formulation has the benefit of providing predictive uncertainty, a property particularly important in scenarios where data is known to be nonstationary.  \nPowerpropagation can impact research within the communities on sparse neural networks and CL. As the method is straightforwardly implemented and poses minimal conceptual or methodological overhead, we anticipate its use for various problems. Furthermore, our use of implicit notions of sparsity has and will likely continue to ","cbCaiijB6CFnAI91","https://ap.wps.com/l/cbCaiijB6CFnAI91","pdf",30790495,1,246,"English","en",105,"# Abstract\n# Impact Statement\n## Functional Regularisation for Continual Learning (FRCL)\n## Powerpropagation and sparse neural networks\n## Meta Learning sparse compression networks\n## Compression for neural data and real-world applications","[{\"question\":\"Why does the thesis focus on efficient machine learning?\",\"answer\":\"Because scaling model size and training data is costly, inference may be limited by hardware, and real applications require continual learning. Efficiency therefore must improve practicality while supporting generalisation and adaptation.\"},{\"question\":\"What core techniques does the thesis introduce?\",\"answer\":\"It develops algorithmic approaches using sparse parameterisations, formalised around explicit or tacit notions of multiple tasks. The goal is to enable tractable computation such as informative data subset selection, model compression without performance loss, and parameters with high plasticity via meta learning.\"},{\"question\":\"How does the thesis connect sparse methods to continual learning and related research?\",\"answer\":\"It discusses Functional Regularisation for Continual Learning (FRCL), including its predictive uncertainty benefits under nonstationary data. It also describes how powerpropagation can support work in sparse neural networks and continual learning.\"}]","Sparse Parameterisations For Efficient Machine Learning Algorithms | PDF",1785682409,620,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":87,"head_meta":89,"extra_data":91,"updated_unix":28},"sparse-parameterisations-for-efficient-machine-learning-algorithms","",{"@graph":36,"@context":86},[37,54,69],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/sparse-parameterisations-for-efficient-machine-learning-algorithms/118227/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":63,"encodingFormat":61,"isAccessibleForFree":64,"interactionStatistic":65},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-05","2026-08-02",true,{"@type":66,"interactionType":67,"userInteractionCount":20},"InteractionCounter",{"@type":68},"ViewAction",{"@type":70,"mainEntity":71},"FAQPage",[72,78,82],{"name":73,"@type":74,"acceptedAnswer":75},"Why does the thesis focus on efficient machine learning?","Question",{"text":76,"@type":77},"Because scaling model size and training data is costly, inference may be limited by hardware, and real applications require continual learning. Efficiency therefore must improve practicality while supporting generalisation and adaptation.","Answer",{"name":79,"@type":74,"acceptedAnswer":80},"What core techniques does the thesis introduce?",{"text":81,"@type":77},"It develops algorithmic approaches using sparse parameterisations, formalised around explicit or tacit notions of multiple tasks. The goal is to enable tractable computation such as informative data subset selection, model compression without performance loss, and parameters with high plasticity via meta learning.",{"name":83,"@type":74,"acceptedAnswer":84},"How does the thesis connect sparse methods to continual learning and related research?",{"text":85,"@type":77},"It discusses Functional Regularisation for Continual Learning (FRCL), including its predictive uncertainty benefits under nonstationary data. It also describes how powerpropagation can support work in sparse neural networks and continual learning.","https://schema.org",{"og:url":52,"og:type":88,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":90,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":93},[94,98,102,106,111,116,121,124,129,132,136],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":95,"show_sort_weight":96,"slug":97},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":99,"show_sort_weight":100,"slug":101},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":103,"show_sort_weight":104,"slug":105},"Exam",70,"exam",{"id":107,"doc_module":4,"doc_module_name":46,"category_name":108,"show_sort_weight":109,"slug":110},5,"Comic",60,"comic",{"id":112,"doc_module":4,"doc_module_name":46,"category_name":113,"show_sort_weight":114,"slug":115},6,"Technology",50,"technology",{"id":117,"doc_module":4,"doc_module_name":46,"category_name":118,"show_sort_weight":119,"slug":120},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":122,"slug":123},30,"research-report",{"id":125,"doc_module":4,"doc_module_name":46,"category_name":126,"show_sort_weight":127,"slug":128},9,"Religion & Spirituality",20,"religion-spirituality",{"id":127,"doc_module":4,"doc_module_name":46,"category_name":130,"show_sort_weight":127,"slug":131},"World Cup","world-cup",{"id":133,"doc_module":4,"doc_module_name":46,"category_name":134,"show_sort_weight":133,"slug":135},10,"Lifestyle","lifestyle",{"id":137,"doc_module":4,"doc_module_name":46,"category_name":138,"show_sort_weight":107,"slug":139},19,"General","general"]