[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"detail-sidebar-cat-0-en-105":3,"doc-seo-128863-105":59,"doc-detail-128863-en":130},{"code":4,"msg":5,"data":6},0,"success",[7,13,18,23,28,33,38,43,48,51,55],{"id":8,"doc_module":4,"doc_module_name":9,"category_name":10,"show_sort_weight":11,"slug":12},1,"Document","Story & Novel",90,"story-novel",{"id":14,"doc_module":4,"doc_module_name":9,"category_name":15,"show_sort_weight":16,"slug":17},2,"Literature",80,"literature",{"id":19,"doc_module":4,"doc_module_name":9,"category_name":20,"show_sort_weight":21,"slug":22},4,"Exam",70,"exam",{"id":24,"doc_module":4,"doc_module_name":9,"category_name":25,"show_sort_weight":26,"slug":27},5,"Comic",60,"comic",{"id":29,"doc_module":4,"doc_module_name":9,"category_name":30,"show_sort_weight":31,"slug":32},6,"Technology",50,"technology",{"id":34,"doc_module":4,"doc_module_name":9,"category_name":35,"show_sort_weight":36,"slug":37},7,"Healthcare",40,"healthcare",{"id":39,"doc_module":4,"doc_module_name":9,"category_name":40,"show_sort_weight":41,"slug":42},8,"Research & Report",30,"research-report",{"id":44,"doc_module":4,"doc_module_name":9,"category_name":45,"show_sort_weight":46,"slug":47},9,"Religion & Spirituality",20,"religion-spirituality",{"id":46,"doc_module":4,"doc_module_name":9,"category_name":49,"show_sort_weight":46,"slug":50},"World Cup","world-cup",{"id":52,"doc_module":4,"doc_module_name":9,"category_name":53,"show_sort_weight":52,"slug":54},10,"Lifestyle","lifestyle",{"id":56,"doc_module":4,"doc_module_name":9,"category_name":57,"show_sort_weight":24,"slug":58},19,"General","general",{"code":4,"msg":60,"data":61},"ok",{"site_id":62,"language":63,"slug":64,"title":65,"keywords":66,"description":67,"schema_data":68,"social_meta":123,"head_meta":125,"extra_data":127,"updated_unix":129},105,"en","on-matching-pursuit-and-coordinate-descent","On Matching Pursuit and Coordinate Descent","","Two widely used first-order optimization approaches on linear spaces—coordinate descent and matching pursuit, including their randomized variants—are analyzed through a shared perspective. By exploiting their connection, a unified affine-invariant theory establishes sublinear O(1/t) rates on smooth convex objectives and linear convergence on strongly convex objectives. The affine-invariant matching-pursuit analysis yields the tightest known rates for steepest coordinate descent. It also provides the first accelerated O(1/t^2) convergence rate for matching pursuit and steepest coordinate descent on convex problems.",{"@graph":69,"@context":122},[70,84,105],{"@type":71,"itemListElement":72},"BreadcrumbList",[73,77,79,82],{"item":74,"name":75,"@type":76,"position":8},"https://docshare.wps.com","Home","ListItem",{"item":78,"name":9,"@type":76,"position":14},"https://docshare.wps.com/document/",{"item":80,"name":40,"@type":76,"position":81},"https://docshare.wps.com/document/research-report/",3,{"item":83,"name":65,"@type":76,"position":19},"https://docshare.wps.com/document/on-matching-pursuit-and-coordinate-descent/128863/",{"url":83,"name":65,"@type":85,"image":86,"author":91,"headline":65,"publisher":94,"fileFormat":97,"inLanguage":63,"description":67,"dateModified":98,"datePublished":99,"encodingFormat":97,"isAccessibleForFree":100,"interactionStatistic":101},"DigitalDocument",{"url":87,"@type":88,"width":89,"height":90},"https://docshare.wps.com/thumbnails/on-matching-pursuit-and-coordinate-descent/128863.png","ImageObject",300,407,{"name":92,"@type":93},"Noah","Person",{"url":74,"name":95,"@type":96},"DocShare","Organization","application/pdf","2026-09-18","2026-08-06",true,{"@type":102,"interactionType":103,"userInteractionCount":44},"InteractionCounter",{"@type":104},"ViewAction",{"@type":106,"mainEntity":107},"FAQPage",[108,114,118],{"name":109,"@type":110,"acceptedAnswer":111},"What optimization problem does the document study?","Question",{"text":112,"@type":113},"It studies minimizing a convex function over a linear space formed by linear combinations of atoms from a set A.","Answer",{"name":115,"@type":110,"acceptedAnswer":116},"How are matching pursuit and coordinate descent connected?",{"text":117,"@type":113},"The document explains that steepest coordinate descent can be viewed as a special case of matching pursuit, and it develops a unified analysis using properties of atomic norms.",{"name":119,"@type":110,"acceptedAnswer":120},"What convergence rates are established for the algorithms?",{"text":121,"@type":113},"It provides affine-invariant sublinear O(1/t) rates on smooth convex objectives, linear convergence on strongly convex objectives, and the first accelerated O(1/t^2) rate on convex objectives for matching pursuit and steepest coordinate descent.","https://schema.org",{"og:url":83,"og:type":124,"og:title":65,"og:site_name":95,"og:description":67},"article",{"robots":126,"canonical":83},"index,follow",{"doc_id":128,"site_id":62},128863,1786004021,{"code":4,"msg":5,"data":131},{"doc_id":128,"user_id":132,"nickname":92,"user_avatar":133,"doc_module":4,"category_id":39,"category_name":40,"doc_title":65,"doc_description":67,"doc_content":134,"file_id":135,"file_url":136,"file_type":137,"file_size":138,"view_count":44,"is_deleted":4,"is_public":8,"is_downloadable":8,"audit_status":8,"page_count":56,"language":139,"language_code":63,"site_id":62,"html_lang":63,"table_of_contents":140,"faqs":141,"seo_title":142,"seo_description":67,"update_tm":129,"read_time":143},137451207643,"https://ap-avatar.wpscdn.com/davatar_3d24733baf745e90a7e4bdd5f77d97b2","View metadata, citation and similar [papers at ](papers at core.ac.uk)[core.ac.uk](papers at core.ac.uk) brought to you by CORE  \nprovided by Infoscience- École polytechnique fédérale de Lausanne  \nOn Matching Pursuit and Coordinate Descent  \nFrancesco Locatello * 1 2 Anant Raj * 1 Sai Praneeth Karimireddy 3 Gunnar Rätsch 2 Bernhard Schölkopf 1 Sebastian U. Stich 3 Martin Jaggi 3  \nAbstract  \nTwo popular examples of first-order optimization methods over linear spaces are coordinate descent and matching pursuit algorithms, with their randomized variants. While the former targets the optimization by moving along coordinates, the latter considers a generalized notion of directions.  \nExploiting the connection between the two algorithms, we present a unified analysis of both, providing affine invariant sublinear O(1=t) rateson smooth objectives and linear convergence on strongly convex objectives. As a byproduct of our affine invariant analysis of matching pursuit, our rates for steepest coordinate descent are the tightest known. Furthermore, we show the first accelerated convergence rate O(1=t2 ) for matching pursuit and steepest coordinate descent on convex objectives.  \n1. Introduction  \nIn this paper we address the following convex optimization problem:  \nmin f (x) ; (1)  \nx2lin(A)  \nwhere f is a convex function. The minimization is over alinear space, which is parametrized as the set of linear combinations of elements from a given set A. These elements of A are called atoms. In the most general setting, A is assumed to be a compact but not necessarily finite subset of a Hilbert space, i.e., a linear space equipped with an inner product, complete in the corresponding norm. Problems of the form (1) are tackled by a multitude of first-order optimization methods and are of paramount interest in the machine learning community (Seber & Lee, 2012 ; Meir &  \n*Equal contribution 1Max Planck Institute for Intelligent Systems 2Dept. of Computer Science, ETH Zurich, Switzerland 3EPFL, Lausanne. Correspondence to: Francesco Locatello \u003C[francesco.locatello@tuebingen.mpg.de](francesco.locatello@tuebingen.mpg.de)>, Anant Raj \u003C[anant.raj@tuebingen.mpg.de](anant.raj@tuebingen.mpg.de)>.  \nProceedings of the 35 th International Conference on Machine Learning, Stockholm, Sweden, PMLR 80, 2018 . Copyright 2018 by the author(s) .  \nRätsch, 2003 ; Schölkopf & Smola, 2001 ; Menard, 2018 ; Tibshirani, 2015) .  \nTraditionally, matching pursuit (MP) algorithms were introduced to solve the inverse problem of representing a measured signal by a sparse combination of atoms from an over-complete basis (Mallat & Zhang, 1993) . In other words, the solution of the optimization problem (1) is formed asa linear combination of few of the elements of the atom set A – i.e. a sparse approximation. At each iteration, the MP algorithm picks a direction from A according to the gradient information, and takes a step. This procedure isnot limited to atoms of fixed dimension. Indeed, lin(A) can be an arbitrary linear subspace of the ambient space and we are interested in finding the minimizer of f only on this domain, see e.g. (Gillis & Luce, 2018) . Conceptually, MP stands in the middle between coordinate descent (CD) and gradient descent, as the algorithm is allowed to descend the function along a prescribed set of directions which does not necessarily correspond to coordinates. This is particularly important for machine learning applications as it translates to a sparse representation of the iterates in terms of the elements of A while maintaining the convergence guarantees (Lacoste-Julien et al., 2013 ; Locatello et al., 2017a) .  \nThe first analysis of the MP algorithm in the optimization sense to solve the template (1) without incoherence assumptions was done by (Locatello et al., 2017b) . To prove convergence, they exploit the connection between MP and the Frank-Wolfe (FW) algorithm (Frank & Wolfe, 1956), a popular projection-free algorithm for the constrained optimization c","cbCaisrD3MluInjd","https://ap.wps.com/l/cbCaisrD3MluInjd","pdf",473615,"English","# Introduction\n## Problem Setup\n## Background and Motivation\n# Contributions and Related Work","[{\"question\":\"What optimization problem does the document study?\",\"answer\":\"It studies minimizing a convex function over a linear space formed by linear combinations of atoms from a set A.\"},{\"question\":\"How are matching pursuit and coordinate descent connected?\",\"answer\":\"The document explains that steepest coordinate descent can be viewed as a special case of matching pursuit, and it develops a unified analysis using properties of atomic norms.\"},{\"question\":\"What convergence rates are established for the algorithms?\",\"answer\":\"It provides affine-invariant sublinear O(1/t) rates on smooth convex objectives, linear convergence on strongly convex objectives, and the first accelerated O(1/t^2) rate on convex objectives for matching pursuit and steepest coordinate descent.\"}]","On Matching Pursuit and Coordinate Descent | PDF",48]