[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-119184-en":3,"doc-seo-119184-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},119184,8796095461610,"Oliver","https://ap-avatar.wpscdn.com/davatar_276721f389ce27ea32af1340a28f341c",8,"Research & Report","Recent and Upcoming Developments in Randomized Numerical Linear Algebra for Machine Learning","Large matrices underpin many machine learning and data analysis tasks, including dataset and graph representations, model weights, and first- and second-order derivatives. Randomized Numerical Linear Algebra (RandNLA) exploits randomness as a computational resource to improve algorithms for common matrix problems. With rising hardware capabilities and efforts to integrate RandNLA into core numerical libraries, new theoretical and practical challenges have emerged. This survey provides a self-contained overview of RandNLA in view of these developments, linking theory to implementation and ML applications.","arXiv :2406 . 11151v2 [ cs .LG] 19 Jun 2024  \nRecent and Upcoming Developments in  \nRandomized Numerical Linear Algebra for Machine Learning  \nMichał Derezi´nski* Michael W. Mahoney†  \nJune 21, 2024  \nAbstract  \nLarge matrices arise in many machine learning and data analysis applications, including as representations of datasets, graphs, model weights, and first and second-order derivatives. Randomized Numerical Linear Algebra (RandNLA) isan area which uses randomness to develop improved algorithms for ubiquitous matrix problems. The area has reached a certain level of maturity; but recent hardware trends, efforts to incorporate RandNLA algorithms into core numerical libraries, and advances in machine learning, statistics, and random matrix theory, have lead to new theoretical and practical challenges. This article provides a self-contained overview of RandNLA, in light of these developments.  \n1 Introduction  \nMatrices provide a natural structure with which to model data. For example, a matrix A ∈ Rm ×n can encode information about m objects, each of which is described by n features. Alternatively, a positive definite matrix A ∈ Rn ×n can encode correlations/similarities between all pairs of n objects. Motivated by large-scale data problems, recent years have witnessed many exciting developments in the theory and practice of matrix algorithms. Particularly remarkable is the use of randomization. Historically, in statistics, machine learning (ML), and domain sciences, randomization has been assumed to be a property of the input data, e.g., due to noise in the data generation mechanisms. In this more recent work on randomization, it is used as an algorithmic or computational resource.  \nRandomized Numerical Linear Algebra (RandNLA) is an interdisciplinary research area that exploits randomizationas a computational resource to develop improved algorithms for large-scale linear algebra problems. From afoundational perspective, it has roots in theoretical computer science (TCS), deep connections with convex analysis, probability theory, and metric embedding theory, etc., as well as strong connections with scientific computing, signal processing, and numerical linear algebra (NLA) . From an implementational perspective, well-engineered RandNLA algorithms beat highly-optimized software libraries for ubiquitous problems such as very over-determined least-squares, they scale well to parallel/distributed environments, and they beat state-of-the-art for a wide range of low-rank matrix approximation problems. From a data analysis perspective, RandNLA has strong connections with ML and statistics and many“non-methodological” applications of data analysis. More generally, of course, it is of continued importance since thereis a growing interest in providing an algorithmic and statistical foundation for modern large-scale data analysis.  \nThe area of RandNLA has achieved a certain level of maturity. As such, there are multiple reviews of the area from multiple different perspectives: introductory overviews (light on prerequisites) [67, 68]; broad and proof-heavy resources [171, 167, 126]; perspectives on interdisciplinary theory (light on proofs) [124, 54]; deep investigations of specific disciplinary topics [94, 104, 131, 132]; and approaches to high-quality software implementations [138] . Particularly notable is the current effort of incorporating RandNLA algorithms into the core numerical libraries (e.g., RandLAPACK and RandBLAS; see [138]) that lie at the foundation of virtually all computational tools in ML (and scientific computing and beyond) .  \nThis level of maturity, as well as recent demands by the ML community and recent trends in hardware, lead to new theoretical and practical challenges that did not exist a decade ago. For example: developing RandLAPACK and  \n* University of Michigan ([derezin@umich.edu](derezin@umich.edu))  \n†ICSI, LBNL, and University of California, Berkeley ([mmahoney@stat.berkeley.edu](mmahoney@stat.berke","cbCaikuSwz6KCecG","https://ap.wps.com/l/cbCaikuSwz6KCecG","pdf",551529,1,29,"English","en",105,"# Introduction\n# Foundations of “Classical” RandNLA\n## Theory up to early developments\n# Ongoing and future trends","[{\"question\":\"What problem does RandNLA address in machine learning and data analysis?\",\"answer\":\"RandNLA targets ubiquitous large-scale matrix problems that arise in tasks such as representing datasets, graphs, model weights, and derivatives.\"},{\"question\":\"How does RandNLA differ from older uses of randomness?\",\"answer\":\"Earlier work often treated randomness as a property of data noise, while RandNLA uses randomness as an algorithmic or computational resource.\"},{\"question\":\"Why have recent developments created new challenges for RandNLA?\",\"answer\":\"Hardware trends and attempts to integrate RandNLA into core numerical libraries, together with advances in neural network training and random matrix theory, require new theory and practical abstractions.\"}]","Recent and Upcoming Developments in Randomized Numerical Linear Algebra for Machine Learning | PDF",1785722978,73,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"recent-and-upcoming-developments-in-randomized-numerical-linear-algebra-for-machine-learning","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/recent-and-upcoming-developments-in-randomized-numerical-linear-algebra-for-machine-learning/119184/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-03",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does RandNLA address in machine learning and data analysis?","Question",{"text":75,"@type":76},"RandNLA targets ubiquitous large-scale matrix problems that arise in tasks such as representing datasets, graphs, model weights, and derivatives.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does RandNLA differ from older uses of randomness?",{"text":80,"@type":76},"Earlier work often treated randomness as a property of data noise, while RandNLA uses randomness as an algorithmic or computational resource.",{"name":82,"@type":73,"acceptedAnswer":83},"Why have recent developments created new challenges for RandNLA?",{"text":84,"@type":76},"Hardware trends and attempts to integrate RandNLA into core numerical libraries, together with advances in neural network training and random matrix theory, require new theory and practical abstractions.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]