[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"detail-sidebar-cat-0-en-105":3,"doc-seo-291206-105":59,"doc-detail-291206-en":131},{"code":4,"msg":5,"data":6},0,"success",[7,13,18,23,28,33,38,43,48,51,55],{"id":8,"doc_module":4,"doc_module_name":9,"category_name":10,"show_sort_weight":11,"slug":12},1,"Document","Story & Novel",90,"story-novel",{"id":14,"doc_module":4,"doc_module_name":9,"category_name":15,"show_sort_weight":16,"slug":17},2,"Literature",80,"literature",{"id":19,"doc_module":4,"doc_module_name":9,"category_name":20,"show_sort_weight":21,"slug":22},4,"Exam",70,"exam",{"id":24,"doc_module":4,"doc_module_name":9,"category_name":25,"show_sort_weight":26,"slug":27},5,"Comic",60,"comic",{"id":29,"doc_module":4,"doc_module_name":9,"category_name":30,"show_sort_weight":31,"slug":32},6,"Technology",50,"technology",{"id":34,"doc_module":4,"doc_module_name":9,"category_name":35,"show_sort_weight":36,"slug":37},7,"Healthcare",40,"healthcare",{"id":39,"doc_module":4,"doc_module_name":9,"category_name":40,"show_sort_weight":41,"slug":42},8,"Research & Report",30,"research-report",{"id":44,"doc_module":4,"doc_module_name":9,"category_name":45,"show_sort_weight":46,"slug":47},9,"Religion & Spirituality",20,"religion-spirituality",{"id":46,"doc_module":4,"doc_module_name":9,"category_name":49,"show_sort_weight":46,"slug":50},"World Cup","world-cup",{"id":52,"doc_module":4,"doc_module_name":9,"category_name":53,"show_sort_weight":52,"slug":54},10,"Lifestyle","lifestyle",{"id":56,"doc_module":4,"doc_module_name":9,"category_name":57,"show_sort_weight":24,"slug":58},19,"General","general",{"code":4,"msg":60,"data":61},"ok",{"site_id":62,"language":63,"slug":64,"title":65,"keywords":66,"description":67,"schema_data":68,"social_meta":124,"head_meta":126,"extra_data":128,"updated_unix":130},105,"en","efcient-proximal-mapping-of-the-1-path-norm-of-shallow-networks","Efficient Proximal Mapping of the 1-path-norm of Shallow Networks","","We demonstrate two new important properties of the 1-path-norm of shallow neural networks. Despite its non-smoothness and non-convexity, it enables a closed-form proximal operator that can be computed efficiently, supporting stochastic proximal-gradient-type methods for regularized empirical risk minimization. When activation functions are differentiable, it provides an upper bound on the network’s Lipschitz constant, tighter than trivial layer-wise products, motivating robust training against adversarial perturbations.",{"@graph":69,"@context":123},[70,84,106],{"@type":71,"itemListElement":72},"BreadcrumbList",[73,77,79,82],{"item":74,"name":75,"@type":76,"position":8},"https://docshare.wps.com","Home","ListItem",{"item":78,"name":9,"@type":76,"position":14},"https://docshare.wps.com/document/",{"item":80,"name":40,"@type":76,"position":81},"https://docshare.wps.com/document/research-report/",3,{"item":83,"name":65,"@type":76,"position":19},"https://docshare.wps.com/document/efcient-proximal-mapping-of-the-1-path-norm-of-shallow-networks/291206/",{"url":83,"name":65,"@type":85,"image":86,"author":91,"headline":65,"publisher":94,"fileFormat":97,"inLanguage":63,"description":67,"dateModified":98,"datePublished":99,"encodingFormat":97,"isAccessibleForFree":100,"interactionStatistic":101},"DigitalDocument",{"url":87,"@type":88,"width":89,"height":90},"https://docshare.wps.com/thumbnails/efcient-proximal-mapping-of-the-1-path-norm-of-shallow-networks/291206.png","ImageObject",300,407,{"name":92,"@type":93},"Valentina","Person",{"url":74,"name":95,"@type":96},"DocShare","Organization","application/pdf","2026-11-08","2026-09-17",true,{"@type":102,"interactionType":103,"userInteractionCount":105},"InteractionCounter",{"@type":104},"ViewAction",12,{"@type":107,"mainEntity":108},"FAQPage",[109,115,119],{"name":110,"@type":111,"acceptedAnswer":112},"What two key properties of the 1-path-norm are established?","Question",{"text":113,"@type":114},"It admits an efficient closed-form proximal operator for stochastic proximal-gradient-style optimization, and—under differentiable activations—provides an upper bound on the Lipschitz constant.","Answer",{"name":116,"@type":111,"acceptedAnswer":117},"Why is the Lipschitz upper bound from the 1-path-norm considered useful?",{"text":118,"@type":114},"The bound is tighter than the trivial layer-wise product of Lipschitz constants, which motivates training networks to be robust to adversarial perturbations.",{"name":120,"@type":111,"acceptedAnswer":121},"How does the work address optimization for the 1-path-norm-regularized objective?",{"text":122,"@type":114},"It proposes an efficient optimization approach based on the new proximal mapping, avoiding issues that can arise when automatic differentiation fails for nonsmooth function compositions.","https://schema.org",{"og:url":83,"og:type":125,"og:title":65,"og:site_name":95,"og:description":67},"article",{"robots":127,"canonical":83},"index,follow",{"doc_id":129,"site_id":62},291206,1789643742,{"code":4,"msg":5,"data":132},{"doc_id":129,"user_id":133,"nickname":92,"user_avatar":134,"doc_module":4,"category_id":39,"category_name":40,"doc_title":65,"doc_description":67,"doc_content":135,"file_id":136,"file_url":137,"file_type":138,"file_size":139,"view_count":105,"is_deleted":4,"is_public":8,"is_downloadable":8,"audit_status":8,"page_count":140,"language":141,"language_code":63,"site_id":62,"html_lang":63,"table_of_contents":142,"faqs":143,"seo_title":144,"seo_description":67,"update_tm":130,"read_time":145,"preview_page_count":4},13056703020460,"https://ap-avatar.wpscdn.com/avatar/be000253dac470eee5d?_k=1778207105932848923","Efficient Proximal Mapping of the 1-path-norm of Shallow Networks  \nFabian Latore 1 Paul Rolland 1 Nadav Hallak 1 Volkan Cevher 1  \nAbstract  \nWe demonstrate two new important properties of the 1-path-norm of shallow neural networks. First, despite its non-smoothness and non-convexity it allows a closed form proximal operator which can be efficiently computed, allowing the use of stochastic proximal-gradient-type methods for regularized empirical risk minimization. Second, when the activation functions is differentiable, it provides an upper bound on the Lipschitz constant of the network. Such bound is tighter than the trivial layer-wise product of Lipschitz constants, motivating its use for training networks robust to adversarial perturbations. In practical experiments we illustrate the advantages of using the proximal mapping and we compare the robustness-accuracy trade-off induced by the 1-path-norm, L1-norm and layer-wise constraints on the Lipschitz constant (Parseval networks) .  \n1. Introduction  \nNeural networks are the backbone of contemporary applications in machine learning and related fields, having huge influence and significance both in theory and practice. Among the most important and desirable attributes of a trained network are robustness and sparsity. Robustness, is often defined as stability to adversarial perturbations, such as in supervised classification methods. The apparent brittleness of neural networks to adversarial attacks in this context has been considered in the literature for some time, see e.g.,(Biggio et al., 2013 ; Szegedy et al., 2013 ; Madry et al., 2018) and references therein.  \nA fundamental question in this regard is how to measure robustness, or more importantly, how to encourage it. One prominent approach supported by theory and practice (Raghunathan et al., 2018 ; Cisse et al., 2017), is to use  \n*Equal contribution 1Learning, information and optimization systems laboratory (LIONS), EPFL, Switzerland. Correspondence to: Fabian Latorre \u003Cfabian.latorre@epfl.ch> .  \nProceedings of the 37th International Conference on Machine Learning, Vienna, Austria, PMLR 119, 2020 . Copyright 2020 by the author(s) .  \nthe Lipschitz constant of the network function to quantize robustness, and regularization to encourage it.  \nThis approach is also supported theoretically with generalization bounds in terms of the layer-wise product of spectral norms (Bartlett et al., 2017 ; Miyato et al., 2018), which particularly upper-bounds the Lipschitz constant. However, a recent empirical study (Jiang* et al., 2020) has found in practice a negative correlation of this measure with generalization. This casts doubts on its usefulness and signals the fact that it is a rather loose upper bound for the Lipschitz constant (Latorre et al., 2020) .  \nCurrent methods that compute upper bounds on the Lipschitz constant of neural networks can be roughly classified into two classes: (i) the class of product bounds, comprising all upper bounds obtained by the multiplication of layer-wise matrix norms; and,(ii) the class of convexoptimization-based bounds, which addresses the network as a whole entity (Raghunathan et al., 2018 ; Fazlyab et al., 2019 ; Latorre et al., 2020) .  \nA trade-off between computational complexity and quality of the upper bound seems apparent. An ideal bound would achieve a balance between both properties: it should provide a good estimate of the constant while being fast and easy to minimize with iterative first-order algorithms.  \nRecently, the path-norm of the network (Neyshabur et al., 2015) has emerged as a complexity measure that is highlycorrelated with generalization (Jiang* et al., 2020) . Thus, its use as a regularizer holds an increasing interest for researchers in the field.  \nDespite existing generalization bounds (Neyshabur et al., 2015), our understanding of the optimization aspects of the path-norm-regularized objective is lacking. Jiang* et al.(2020) refrained from using automatic-differentiation ","cbCair0WQwViL254","https://ap.wps.com/l/cbCair0WQwViL254","pdf",1142910,28,"English","# Introduction\n## Robustness and sparsity in neural networks\n## Measuring robustness via Lipschitz constants and regularization\n## Existing upper-bound methods: product vs convex-optimization\n## Path-norm as a complexity measure\n## Optimization challenges and main contributions\n## Optimization: proximal-gradient with 1-path-norm","[{\"question\":\"What two key properties of the 1-path-norm are established?\",\"answer\":\"It admits an efficient closed-form proximal operator for stochastic proximal-gradient-style optimization, and—under differentiable activations—provides an upper bound on the Lipschitz constant.\"},{\"question\":\"Why is the Lipschitz upper bound from the 1-path-norm considered useful?\",\"answer\":\"The bound is tighter than the trivial layer-wise product of Lipschitz constants, which motivates training networks to be robust to adversarial perturbations.\"},{\"question\":\"How does the work address optimization for the 1-path-norm-regularized objective?\",\"answer\":\"It proposes an efficient optimization approach based on the new proximal mapping, avoiding issues that can arise when automatic differentiation fails for nonsmooth function compositions.\"}]","Efficient Proximal Mapping of the 1-path-norm of Shallow Networks | PDF",71]