[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84453-en":3,"doc-seo-84453-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},84453,1099513958762,"Logic","https://ap-avatar.wpscdn.com/avatar/1000023916a998db790?x-image-process=image/resize,m_fixed,w_180,h_180&k=1782109480056885918",8,"Research & Report","An Elementary Proof of the Near Optimality of LogSumExp Smoothing","The paper studies smoothings of the coordinate-wise max function on Rd under the infinity norm, focusing on convex overestimators and their approximation error. The classical LogSumExp (softmax) provides a ln(d)-smoothing with a tight uniform bound to the max. An elementary construction yields a lower bound showing any convex, overestimating smoothing must differ by at least about 0.8145 ln(d). LogSumExp is optimal up to constants, though stronger entropy-based variants are shown not to be exactly optimal; exact optimal smoothings are obtained in small dimensions.","arXiv :2512 . 10825v3 [math . ST] 13 Jul 2026  \nAn Elementary Proof of the Near Optimality of LogSumExp Smoothing  \nThabo Samakhoana∗ Benjamin Grimmer†  \nAbstract  \nWe consider the design of smoothings of the (coordinate-wise) max function in Rd in the infinity norm. The LogSumExp function f (x) = ln(Pdi exp(xi )) provides a classical smoothing, differing from the max function in value by at most ln (d) . We provide an elementary construction of a lower bound, establishing that every overestimating smoothing of the max function must differ by at least ∼ 0.8145ln(d) . Hence, LogSumExp is optimal up to small constant factors. However, we provide strictly stronger smoothings showing the entropy-based LogSumExp approach is not exactly optimal. In small dimensions, we propose exactly optimal smoothings, attaining our lower bound.  \n1 Introduction  \nWe consider the task of smoothing the coordinate-wise max function σmax(x) = maxi=1 , ...,d xi with respect to the infinity norm in d dimensions. Such smoothings play an important role in the acceleration of nonsmooth optimization algorithms [1] and recently in machine learning [2, 3] . In both of these settings, the function fLSE (x) = ln(Pdi exp (xi)) has played a central role as the default“best” choice in the engineering of algorithms.  \nThe canonical choice of the LogSumExp function fLSE (also known as the “softmax”) is a convex overestimator of σmax that is 1-smooth with respect to the infinity norm:  \n∥∇f(x) − ∇f(y)∥ 1 ≤ ∥x − y∥∞ ∀x, y ∈ Rd. (1 . 1)  \nMoreover, fLSE closely approximates the max function, having  \n∥fLSE − σmax∥∞ = ln(d)  \nwhere ∥f − σ∥∞ := sup x∈Rd |f(x) − σ(x)| . Generally, we say that f is a δ-smoothing with respect to the infinity norm of the max function if f is convex and 1-smooth w.r.t. ∥·∥∞ with ∥f −σmax∥∞ ≤ δ . Hence, fLSE is a ln(d)-smoothing of σmax.  \nSuch δ-smoothings in the infinity norm play an important role for both of the applications mentioned above. Below we discuss these two core motivations:  \n(i) In nonsmooth optimization, a common form for a problem to take is minimizing a maximum of convex functions  \nmin max gi(y) = σmax ◦ g(y)  \ny∈Rn i=1,...,d  \n∗ Johns Hopkins University, Department of Applied Mathematics and Statistics, [tsamakh1@jhu.edu](tsamakh1@jhu.edu)[ ](tsamakh1@jhu.edu)†Johns Hopkins University, Department of Applied Mathematics and Statistics, [grimmer@jhu.edu](grimmer@jhu.edu)  \nwhere g (y) = [g1 (y), ... , gd(y)]T . For general convex nonsmooth optimization, any first-order subgradient method seeking an ϵ-minimizer requires at least Ω(1/ϵ2 ) subgradient oracle queries. However, if each individual component objective gi is M-Lipschitz and L-smooth with respect to the Euclidean norm in Rn, then accelerated optimization algorithms from smooth convex optimization can be leveraged. In particular, [4] considered minimizing the smooth convex relaxation  \ninRn 􀀐 ϵ2δ f 􀀑◦ 􀀒 2δϵ g 􀀓 (y)  \nfor a δ-smoothing f of σmax. Since f is 1-smooth with respect to the infinity norm and each individual gi is smooth and Lipschitz, the objective y →7 ~~ϵ~~2δf ◦ ~~2δ~~ϵg (y) above must be (L + 2δM2 /ϵ)-smooth with respect to the Euclidean norm in Rn [4, Proposition 4.1] . Further, it differs from the original objective by at most ϵ/2 . As a result, by applying an accelerated algorithm for smooth convex optimization like Nesterov’s method [5] to produce an ϵ/2-minimizer, an ϵ-minimizer to the original problem can be produced with a first-order oracle complexity of at most  \ns ~~ ~~(L~~ ~~+~~ ~~2δM2/ϵ/ϵ2)~~ ~~∥y0~~ ~~−~~ ~~y⋆∥2 = O ~~ ~~√Lϵ~~ ~~+~~ ~~δMϵ2~~ ~~∥y0~~ ~~−~~ ~~y⋆∥! . (1.2)  \nNote the acceleration in the dependence on ϵ in the above O(1/ϵ) rate over the general Ω(1/ϵ2 ) limit. Hence, δ-smoothings provide a direct reduction for accelerating structured nonsmooth optimization. Further, improvements in δ directly improve oracle complexities.  \n(ii) In neural network design, especially modern networks using the attention/transformer mechanism [6], ","cbCaif2GeSnzdpDm","https://ap.wps.com/l/cbCaif2GeSnzdpDm","pdf",588610,1,15,"English","en",105,"# Introduction\n## Smoothings of the max function and LogSumExp\n## Applications: nonsmooth optimization acceleration\n## Applications: neural network attention/softmax interpretation\n## Dual perspective via convex conjugates and entropy","[{\"question\":\"What function and norm does the paper focus on when analyzing smoothing?\",\"answer\":\"It smooths the coordinate-wise max function in Rd with respect to the infinity norm, using convex overestimators that approximate the max while retaining smoothness properties.\"},{\"question\":\"Why is LogSumExp considered a canonical smoothing for the max function?\",\"answer\":\"LogSumExp is a convex overestimator that is 1-smooth with respect to the infinity norm and approximates the max with uniform error at most ln(d).\"},{\"question\":\"What is the main lower-bound result about overestimating smoothings?\",\"answer\":\"The paper provides an elementary lower bound showing every overestimating smoothing must differ from the max by at least approximately 0.8145 ln(d), implying LogSumExp is near-optimal up to constant factors.\"}]",1784195711,38,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"an-elementary-proof-of-the-near-optimality-of-logsumexp-smoothing","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/an-elementary-proof-of-the-near-optimality-of-logsumexp-smoothing/84453/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What function and norm does the paper focus on when analyzing smoothing?","Question",{"text":75,"@type":76},"It smooths the coordinate-wise max function in Rd with respect to the infinity norm, using convex overestimators that approximate the max while retaining smoothness properties.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"Why is LogSumExp considered a canonical smoothing for the max function?",{"text":80,"@type":76},"LogSumExp is a convex overestimator that is 1-smooth with respect to the infinity norm and approximates the max with uniform error at most ln(d).",{"name":82,"@type":73,"acceptedAnswer":83},"What is the main lower-bound result about overestimating smoothings?",{"text":84,"@type":76},"The paper provides an elementary lower bound showing every overestimating smoothing must differ from the max by at least approximately 0.8145 ln(d), implying LogSumExp is near-optimal up to constant factors.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]