[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-128862-en":3,"detail-sidebar-cat-0-en-105":31,"doc-seo-128862-105":79},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":28,"seo_description":14,"update_tm":29,"read_time":30},128862,137451207643,"Noah","https://ap-avatar.wpscdn.com/davatar_3d24733baf745e90a7e4bdd5f77d97b2",8,"Research & Report","Optimal Rates of Sketched-regularized Algorithms for Least-Squares Regression over Hilbert Spaces","Investigate regularized least-squares regression in a Hilbert space using projection-based methods, including nonparametric learning over reproducing kernel Hilbert spaces. Prove convergence under norm variants, assuming a capacity condition on the hypothesis space and a regularity condition on the target function. Derive optimal, distribution-dependent rates for randomized sketched-regularized algorithms, requiring sketch dimension proportional to the effective dimension up to log factors. Obtain parallel results for Nyström regularization and show no saturation effects in both attainable and non-attainable regimes.","CORE  Metadata, citation and similar [papers at core.ac.uk](papers at core.ac.uk)  \nProvided by Infoscience- École polytechnique fédérale de Lausanne  \nOptimal Rates of Sketched-regularized Algorithms for Least-Squares  \nRegression over Hilbert Spaces  \nJunhong Lin 1 Volkan Cevher 1  \nAbstract  \nWe investigate regularized algorithms combining with projection for least-squares regression problem over a Hilbert space, covering nonparametric regression over a reproducing kernel Hilbert space. We prove convergence results with respect to variants of norms, under a capacity assumption on the hypothesis space and a regularity condition on the target function. As a result, we obtain optimal rates for regularized algorithms with randomized sketches, provided that the sketch dimension is proportional to the effective dimension up to a logarithmic factor. As a byproduct, we obtain similar results for Nystrm regularized algorithms. Our results are the first ones with optimal, distribution-dependent rates that do not have any saturation effect for sketched/Nystrm regularized algorithms, considering both the attainable and non-attainable cases.  \n1. Introduction  \nLet the input space H be a separable Hilbert space with inner product denoted by h􀀁 ; 􀀁iH , and the output space R. Let 􀀚 be an unknown probability measure on H 􀀂 R. In this paper, we study the following expected risk minimization,  \ninf  \n!2H  \n(!); (!) = ZH􀀂R(h!; xiH 􀀀 y)2 d􀀚 (x; y); (1)  \nwhere the measure 􀀚 is known only through a sample z = fzi = (xi ; yi)g of size n 2 N, independently and identically distributed (i.i.d.) according to 􀀚 .  \nThe above regression setting covers nonparametric regression over a reproducing kernel Hilbert space (Cucker & Zhou, 2007; Steinwart & Christmann, 2008), and it is close  \n1 Laboratory for Information and Inference Systems, ´Ecole Polytechnique Fdrale de Lausanne, Lausanne, Switzerland. Correspondence to: Junhong Lin \u003Cjunhong.lin@epfl.ch>, Volkan Cevher \u003Cvolkan.cevher@epfl.ch> .  \nProceedings of the 35 th International Conference on Machine Learning, Stockholm, Sweden, PMLR 80, 2018 . Copyright 2018 by the author(s) .  \nto functional regression (Ramsay, 2006) and linear inverse problems (Engl et al., 1996) . A basic algorithm for the problem is ridge regression, and its generalization, spectralregularized algorithm. Such algorithms can be viewed as solving an empirical, linear equation with the empirical covariance operator replaced by a regularized one, see (Caponnetto & Yao, 2006; Bauer et al., 2007; Gerfo et al., 2008; Linet al., 2018) and references therein. Here, the regularizationis used to control the complexity of the solution to against over-fitting and to achieve best generalization ability. The function/estimator generated by classic regularized algorithm is in the subspace spanfxg of H , where x = fx1 ; 􀀁 􀀁 􀀁 ; xng: More often, the search of an estimator for some specific algorithms is restricted to a different (and possibly smaller) subspace S, which leads to regularized algorithms with projection. Such approaches have computational advantages in nonparametric regression with kernel methods (Williams & Seeger, 2000; Smola & Schlkopf, 2000) . Typically, with a subsample/sketch dimension m \u003C n, S = spanfj : 1 􀀔 j 􀀔 mg where j  is chosen randomly from the input set x, or S = spanfP Gij xj : 1 􀀔 i 􀀔 mg where G =[Gij]1􀀔i􀀔m;1􀀔j􀀔n is a general randomized matrix whose rows are drawn according to a distribution. The resulted algorithms are called Nystrm regularized algorithm and sketched-regularized algorithm, respectively.  \nOur starting points of this paper are recent papers (Bach, 2013; Alaoui & Mahoney, 2015; Yang et al., 2017; Rudiet al., 2015; Myleiko et al., 2017) where convergence resultson Nystrm/sketched regularized algorithms for learning with kernel methods are given. Particularly, within the fixed design setting, i.e., the input set x are deterministic while the output set y = fy1 ; 􀀁 􀀁 􀀁 ; yng treated randomly, convergence res","cbCait5mJNvQY07U","https://ap.wps.com/l/cbCait5mJNvQY07U","pdf",408984,9,1,20,"English","en",105,"# Introduction\n## Problem setting and expected risk minimization\n## Regularized and projection-based algorithms\n## Sketching and Nyström approaches\n## Motivation and contributions","[{\"question\":\"What regression problem setting is studied in the paper?\",\"answer\":\"The paper studies expected risk minimization for least-squares regression over a separable Hilbert space, where data are sampled i.i.d. from an unknown probability measure.\"},{\"question\":\"What do sketch dimension and effective dimension need to satisfy for optimal rates?\",\"answer\":\"Optimal rates for randomized sketched-regularized algorithms hold when the sketch dimension is proportional to the effective dimension up to a logarithmic factor.\"},{\"question\":\"Does the paper address both attainable and non-attainable cases?\",\"answer\":\"Yes. The results target both attainable and non-attainable regimes and emphasize that the derived rates do not suffer from saturation effects.\"}]","Optimal Rates of Sketched-regularized Algorithms for Least-Squares Regression over Hilbert Spaces | PDF",1786004020,50,{"code":4,"msg":5,"data":32},[33,38,43,48,53,57,62,65,68,71,75],{"id":21,"doc_module":4,"doc_module_name":34,"category_name":35,"show_sort_weight":36,"slug":37},"Document","Story & Novel",90,"story-novel",{"id":39,"doc_module":4,"doc_module_name":34,"category_name":40,"show_sort_weight":41,"slug":42},2,"Literature",80,"literature",{"id":44,"doc_module":4,"doc_module_name":34,"category_name":45,"show_sort_weight":46,"slug":47},4,"Exam",70,"exam",{"id":49,"doc_module":4,"doc_module_name":34,"category_name":50,"show_sort_weight":51,"slug":52},5,"Comic",60,"comic",{"id":54,"doc_module":4,"doc_module_name":34,"category_name":55,"show_sort_weight":30,"slug":56},6,"Technology","technology",{"id":58,"doc_module":4,"doc_module_name":34,"category_name":59,"show_sort_weight":60,"slug":61},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":34,"category_name":12,"show_sort_weight":63,"slug":64},30,"research-report",{"id":20,"doc_module":4,"doc_module_name":34,"category_name":66,"show_sort_weight":22,"slug":67},"Religion & Spirituality","religion-spirituality",{"id":22,"doc_module":4,"doc_module_name":34,"category_name":69,"show_sort_weight":22,"slug":70},"World Cup","world-cup",{"id":72,"doc_module":4,"doc_module_name":34,"category_name":73,"show_sort_weight":72,"slug":74},10,"Lifestyle","lifestyle",{"id":76,"doc_module":4,"doc_module_name":34,"category_name":77,"show_sort_weight":49,"slug":78},19,"General","general",{"code":4,"msg":80,"data":81},"ok",{"site_id":25,"language":24,"slug":82,"title":13,"keywords":83,"description":14,"schema_data":84,"social_meta":138,"head_meta":140,"extra_data":142,"updated_unix":29},"optimal-rates-of-sketched-regularized-algorithms-for-least-squares-regression-over-hilbert-spaces","",{"@graph":85,"@context":137},[86,100,120],{"@type":87,"itemListElement":88},"BreadcrumbList",[89,93,95,98],{"item":90,"name":91,"@type":92,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":94,"name":34,"@type":92,"position":39},"https://docshare.wps.com/document/",{"item":96,"name":12,"@type":92,"position":97},"https://docshare.wps.com/document/research-report/",3,{"item":99,"name":13,"@type":92,"position":44},"https://docshare.wps.com/document/optimal-rates-of-sketched-regularized-algorithms-for-least-squares-regression-over-hilbert-spaces/128862/",{"url":99,"name":13,"@type":101,"image":102,"author":107,"headline":13,"publisher":109,"fileFormat":112,"inLanguage":24,"description":14,"dateModified":113,"datePublished":114,"encodingFormat":112,"isAccessibleForFree":115,"interactionStatistic":116},"DigitalDocument",{"url":103,"@type":104,"width":105,"height":106},"https://docshare.wps.com/thumbnails/optimal-rates-of-sketched-regularized-algorithms-for-least-squares-regression-over-hilbert-spaces/128862.png","ImageObject",300,407,{"name":9,"@type":108},"Person",{"url":90,"name":110,"@type":111},"DocShare","Organization","application/pdf","2026-09-18","2026-08-06",true,{"@type":117,"interactionType":118,"userInteractionCount":20},"InteractionCounter",{"@type":119},"ViewAction",{"@type":121,"mainEntity":122},"FAQPage",[123,129,133],{"name":124,"@type":125,"acceptedAnswer":126},"What regression problem setting is studied in the paper?","Question",{"text":127,"@type":128},"The paper studies expected risk minimization for least-squares regression over a separable Hilbert space, where data are sampled i.i.d. from an unknown probability measure.","Answer",{"name":130,"@type":125,"acceptedAnswer":131},"What do sketch dimension and effective dimension need to satisfy for optimal rates?",{"text":132,"@type":128},"Optimal rates for randomized sketched-regularized algorithms hold when the sketch dimension is proportional to the effective dimension up to a logarithmic factor.",{"name":134,"@type":125,"acceptedAnswer":135},"Does the paper address both attainable and non-attainable cases?",{"text":136,"@type":128},"Yes. The results target both attainable and non-attainable regimes and emphasize that the derived rates do not suffer from saturation effects.","https://schema.org",{"og:url":99,"og:type":139,"og:title":13,"og:site_name":110,"og:description":14},"article",{"robots":141,"canonical":99},"index,follow",{"doc_id":7,"site_id":25}]