[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-117638-en":3,"doc-seo-117638-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},117638,13056703019662,"Evangeline","https://ap-avatar.wpscdn.com/avatar/be000253a8e92610077?_k=1778726343310543188",8,"Research & Report","Regularized Bundle Methods for Convex and Non-Convex Risks","Machine learning is commonly formulated as optimization, where convex objectives enable efficient convex solvers and favorable guarantees. However, non-convexity is widespread in practice and forcing convexity can restrict modeling expressiveness. This work develops efficient, scalable algorithms for non-convex optimization by studying regularized unconstrained problems covering logistic regression, conditional random fields, and large-margin estimation. A cutting-plane based method exploits regularization to handle convex and non-convex, smooth and non-smooth risks, proving convergence for convex risks and validating performance empirically.","CORE  Metadata, citation and similar [papers at core.ac.uk](papers at core.ac.uk)  \nProvided by Infoscience- École polytechnique fédérale de Lausanne  \nJournal of Machine Learning Research 13 (2012) 3539-3583 Submitted 4/10; Revised 4/12; Published 12/12  \nRegularized Bundle Methods for Convex and Non-Convex Risks  \nTrinh-Minh-Tri Do∗ TRI. DO@IDIAP. CH  \nIdiap Research Institute Rue Marconi 19  \n1920 Martigny, Switzerland  \nThierry Artires THIERRY. ARTIERES@LIP 6.FR  \nLIP6-Universite´ Pierre et Marie Curie  \n104 avenue dupre´sident Kennedy  \n75016 Paris, France  \nEditor: Tony Jebara  \nAbstract  \nMachine learning is most often cast as an optimization problem. Ideally, one expects a convex objective function to rely on ef􀀂cient convex optimizers with nice guarantees such as no local optima.  \nYet, non-convexity is very frequent in practice and it may sometimes be inappropriate to look for convexity at any price. Alternatively one can decide not to limit a priori the modeling expressivity to models whose learning may be solved by convex optimization and rely on non-convex optimization algorithms. The main motivation of this work is to provide ef􀀂cient and scalable algorithms for non-convex optimization. We focus on regularized unconstrained optimization problems which cover a large number of modern machine learning problems such as logistic regression, conditional random 􀀂elds, large margin estimation, etc. We propose a novel algorithm for minimizing a regularized objective that is able to handle convex and non-convex, smooth and non-smooth risks. The algorithm is based on the cutting plane technique and on the idea of exploiting the regularization term in the objective function. It may be thought as a limited memory extension of convex regularized bundle methods for dealing with convex and non convex risks. In case the risk is convex the algorithm is proved to converge to a stationary solution with accuracy ε with a rate O (1/λε) where λ is the regularization parameter of the objective function under the assumption of a Lipschitz empirical risk. In case the risk is not convex getting such a proof is more dif􀀂cult and requires a stronger and more disputable assumption. Yet we provide experimental results on arti􀀂cial test problems, and on 􀀂ve standard and dif􀀂cult machine learning problems that are cast as convex and non-convex optimization problems that show how our algorithm compares well in practice with state of the art optimization algorithms.  \nKeywords: optimization, non-convex, non-smooth, cutting plane, bundle method, regularized risk  \n1. Introduction  \nMachine learning is most often cast as an optimization problem where one looks for the best model among a parameterized family of models. The best model is de􀀂ned as the one with the set of parameters that minimizes an objective function (i.e. criterion) . For some years now machine learning community aimed at designing new models in such a way that the resulting objective function is convex. Doing so brings the fundamental advantage that one can rely on ef􀀂cient convex optimiza-  \n∗ . Part of this work was done when TMT Do was at LIP6 .  \n􀀍c2012 Trinh Minh Tri Do and Thierry Artires.  \nDO AND ARTIRES  \ntion algorithms, with nice guarantees such as no local optima and easier theoretical analysis (e.g. for the convergence rate) . For instance logistic regression, support vector machine, maximum margin Markov network, and conditional random 􀀂elds have found widespread use in basic machine learning applications.  \nHowever, such a “simple convex modeling” may actually be outperformed by non-convex modeling in some important applications. For example on MNIST database, convex Gaussian-SVM reaches 1 .4% error rate vs. 0.53% for non-convex convolutional nets (Jarrett et al., 2009) .1 Also non-convexity is much more frequent than convexity “in real life”. A number of problems that machine learning researchers face today may not be easily cast as convex optimization problems with","cbCaib7nelTYoPKr","https://ap.wps.com/l/cbCaib7nelTYoPKr","pdf",578581,1,45,"English","en",105,"# Introduction\n## Convex optimization vs. non-convex modeling\n## Strategies for handling non-convexity in learning\n## Convex relaxation and its limitations\n## Direct non-convex optimization approaches\n## Regularized bundle methods in the proposed framework","[{\"question\":\"Why do the authors argue that convex modeling is not always sufficient in machine learning?\",\"answer\":\"Non-convexity is frequent in real applications and may be hard to cast as a convex problem without limiting model expressivity. Forcing convexity can therefore hurt learning and empirical performance.\"},{\"question\":\"What problem setting does the paper focus on?\",\"answer\":\"Regularized unconstrained optimization problems, which include many modern machine learning tasks such as logistic regression, conditional random fields, and large margin estimation.\"},{\"question\":\"How does the proposed algorithm handle both convex and non-convex risks?\",\"answer\":\"It is based on the cutting plane technique and explicitly exploits the regularization term in the objective, enabling it to manage convex and non-convex as well as smooth and non-smooth risks.\"}]","Regularized Bundle Methods for Convex and Non-Convex Risks | PDF",1785677536,113,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"regularized-bundle-methods-for-convex-and-non-convex-risks","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/regularized-bundle-methods-for-convex-and-non-convex-risks/117638/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-02",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why do the authors argue that convex modeling is not always sufficient in machine learning?","Question",{"text":75,"@type":76},"Non-convexity is frequent in real applications and may be hard to cast as a convex problem without limiting model expressivity. Forcing convexity can therefore hurt learning and empirical performance.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What problem setting does the paper focus on?",{"text":80,"@type":76},"Regularized unconstrained optimization problems, which include many modern machine learning tasks such as logistic regression, conditional random fields, and large margin estimation.",{"name":82,"@type":73,"acceptedAnswer":83},"How does the proposed algorithm handle both convex and non-convex risks?",{"text":84,"@type":76},"It is based on the cutting plane technique and explicitly exploits the regularization term in the objective, enabling it to manage convex and non-convex as well as smooth and non-smooth risks.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]