[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-128708-en":3,"doc-seo-128708-105":31,"detail-sidebar-cat-0-en-105":92},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":28,"seo_description":14,"update_tm":29,"read_time":30},128708,962084926284,"Aurora","https://ap-avatar.wpscdn.com/davatar_29158cc5080c5b710cf443261637dec0",8,"Research & Report","Primal-Dual Rates and Certificates - Convergence Guarantees for Optimization Methods","We propose an algorithm-independent framework that equips existing optimization methods with primal-dual certificates and explicit rates of convergence. Such certificates enable practitioners to diagnose optimization progress, especially in machine learning where the optimum is typically unknown. The work derives new primal-dual convergence rates for Lasso, L1 regularized variants, Elastic Net, group Lasso, and TV-regularized objectives, and shows the theory applies to norm-regularized generalized linear models. It provides efficiently computable, globally defined duality gaps without altering the original problems in the region of interest.","View metadata, citation and similar [papers at ](papers at core.ac.uk)[core.ac.uk](papers at core.ac.uk) brought to you by CORE  \nprovided by Infoscience- École polytechnique fédérale de Lausanne  \nPrimal-Dual Rates and Certificates  \nCelestine Dnner CDU @ZURICH . IBM . COM  \nIBM Research, Z¨urich, Switzerland  \nSimone Forte FORTESIMONE 90@ GMAIL . COM  \nETH Z¨urich, Switzerland  \nMartin Tak TAKAC . MT@GMAIL . COM  \nLehigh University, USA  \nMartin Jaggi JAGGIM @INF. ETHZ . CH  \nETH Z¨urich, Switzerland  \nAbstract  \nWe propose an algorithm-independent framework to equip existing optimization methods with primal-dual certificates. Such certificatesand corresponding rate of convergence guarantees are important for practitioners to diagnose progress, in particular in machine learning applications.  \nWe obtain new primal-dual convergence rates, e.g., for the Lasso as well as many L 1 , Elastic Net, group Lasso and TV-regularized problems. The theory applies to any norm-regularized generalized linear model. Our approach provides efficiently computable duality gaps which are globally defined, without modifying the original problems in the region of interest.  \n1. Introduction  \nThe massive growth of available data has moved data analysis and machine learning to center stage in many industrial as well as scientific fields, ranging from web and sensor data to astronomy, health science, and countless other applications. With the increasing size of datasets, machine learning methods are limited by the scalability of the underlying optimization algorithms to train these models, which has spurred significant research interest in recent years.  \nHowever, practitioners face a significant problem arising with the larger model complexity in large-scale machine learning and in particular deep-learning methods-it is increasingly hard to diagnose if the optimization algorithm  \nProceedings of the 33 rd International Conference on Machine Learning, New York, NY, USA, 2016 . JMLR: W&CP volume 48. Copyright 2016 by the author(s) .  \nused for training works well or not. With the optimization algorithms also becoming more complex (e.g., in a distributed setting), it can often be very hard to pin down if bad performance of a predictive model either comes from slow optimization, or from poor modeling choices. In this light, easily verifiable guarantees for the quality of an optimization algorithm are very useful—note that the optimum solution of the problem is unknown in most cases. For convex optimization problems, a primal-dual gap can serve as such a certificate. If available, the gap also serves as a useful stopping criterion for the optimizer.  \nSo far, the majority of popular optimization algorithms for learning applications comes without a notion of primaldual gap. In this paper, we aim to change this for a relevant class of machine learning problems. We propose a primaldual framework which is algorithm-independent, and allows to equip existing algorithms with additional primaldual certificates as an add-on.  \nOur approach is motivated by the recent analysis of SDCA (Shalev-Shwartz & Zhang, 2013) . We extend their setting to the significantly larger class of convex optimization problems of the form  \nmin  \n􀀋2Rn  \nfor a given matrix A 2  \nf (A􀀋) + g (􀀋)  \nRd􀀂n , f being smooth, and g be-  \ning a general convex function. This problem class includes the most prominent regression and classification methods as well as generalized linear models. We will formalize the setting in more details in Section 3, and highlight the associated dual problem, which has the same structure. An overview over some popular examples that can be formulated in this setting either as primal or dual problem is given in Table 1.  \nContributions. The main contributions in this work can be summarized as follows:  \n􀀏 Our new primal-dual framework is algorithmindependent, that is it allows users to equip existing algorithms with primal-dual certificates and convergence rates.  \n􀀏 We introduce a","cbCaijPFQbrFA65e","https://ap.wps.com/l/cbCaijPFQbrFA65e","pdf",860329,3,1,23,"English","en",105,"# Abstract\n# 1. Introduction\n# 2. Related Work","[{\"question\":\"What problem does the paper address?\",\"answer\":\"It targets the difficulty practitioners face in diagnosing whether optimization algorithms are performing well in large-scale machine learning, where the optimal solution is usually unknown.\"},{\"question\":\"What does the proposed framework provide?\",\"answer\":\"It introduces an algorithm-independent way to add primal-dual certificates and corresponding convergence rates to existing optimization methods, including efficiently computable duality gaps.\"},{\"question\":\"Which machine learning problems and models does the theory cover?\",\"answer\":\"The results include Lasso and many L1-type regularized problems such as Elastic Net, group Lasso, and TV-regularized objectives, and it applies to norm-regularized generalized linear models.\"}]","Primal-Dual Rates and Certificates - Convergence Guarantees for Optimization Methods | PDF",1786002788,58,{"code":4,"msg":32,"data":33},"ok",{"site_id":25,"language":24,"slug":34,"title":13,"keywords":35,"description":14,"schema_data":36,"social_meta":87,"head_meta":89,"extra_data":91,"updated_unix":29},"primal-dual-rates-and-certificates-convergence-guarantees-for-optimization-methods","",{"@graph":37,"@context":86},[38,54,69],{"@type":39,"itemListElement":40},"BreadcrumbList",[41,45,49,51],{"item":42,"name":43,"@type":44,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":46,"name":47,"@type":44,"position":48},"https://docshare.wps.com/document/","Document",2,{"item":50,"name":12,"@type":44,"position":20},"https://docshare.wps.com/document/research-report/",{"item":52,"name":13,"@type":44,"position":53},"https://docshare.wps.com/document/primal-dual-rates-and-certificates-convergence-guarantees-for-optimization-methods/128708/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":24,"description":14,"dateModified":62,"datePublished":63,"encodingFormat":61,"isAccessibleForFree":64,"interactionStatistic":65},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":42,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-24","2026-08-06",true,{"@type":66,"interactionType":67,"userInteractionCount":20},"InteractionCounter",{"@type":68},"ViewAction",{"@type":70,"mainEntity":71},"FAQPage",[72,78,82],{"name":73,"@type":74,"acceptedAnswer":75},"What problem does the paper address?","Question",{"text":76,"@type":77},"It targets the difficulty practitioners face in diagnosing whether optimization algorithms are performing well in large-scale machine learning, where the optimal solution is usually unknown.","Answer",{"name":79,"@type":74,"acceptedAnswer":80},"What does the proposed framework provide?",{"text":81,"@type":77},"It introduces an algorithm-independent way to add primal-dual certificates and corresponding convergence rates to existing optimization methods, including efficiently computable duality gaps.",{"name":83,"@type":74,"acceptedAnswer":84},"Which machine learning problems and models does the theory cover?",{"text":85,"@type":77},"The results include Lasso and many L1-type regularized problems such as Elastic Net, group Lasso, and TV-regularized objectives, and it applies to norm-regularized generalized linear models.","https://schema.org",{"og:url":52,"og:type":88,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":90,"canonical":52},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":93},[94,98,102,106,111,116,121,124,129,132,136],{"id":21,"doc_module":4,"doc_module_name":47,"category_name":95,"show_sort_weight":96,"slug":97},"Story & Novel",90,"story-novel",{"id":48,"doc_module":4,"doc_module_name":47,"category_name":99,"show_sort_weight":100,"slug":101},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":47,"category_name":103,"show_sort_weight":104,"slug":105},"Exam",70,"exam",{"id":107,"doc_module":4,"doc_module_name":47,"category_name":108,"show_sort_weight":109,"slug":110},5,"Comic",60,"comic",{"id":112,"doc_module":4,"doc_module_name":47,"category_name":113,"show_sort_weight":114,"slug":115},6,"Technology",50,"technology",{"id":117,"doc_module":4,"doc_module_name":47,"category_name":118,"show_sort_weight":119,"slug":120},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":47,"category_name":12,"show_sort_weight":122,"slug":123},30,"research-report",{"id":125,"doc_module":4,"doc_module_name":47,"category_name":126,"show_sort_weight":127,"slug":128},9,"Religion & Spirituality",20,"religion-spirituality",{"id":127,"doc_module":4,"doc_module_name":47,"category_name":130,"show_sort_weight":127,"slug":131},"World Cup","world-cup",{"id":133,"doc_module":4,"doc_module_name":47,"category_name":134,"show_sort_weight":133,"slug":135},10,"Lifestyle","lifestyle",{"id":137,"doc_module":4,"doc_module_name":47,"category_name":138,"show_sort_weight":107,"slug":139},19,"General","general"]