[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-85977-en":3,"doc-seo-85977-105":30,"detail-sidebar-cat-0-en-105":92},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":13,"seo_description":14,"update_tm":28,"read_time":29},85977,1374391975076,"Riley","https://ap-avatar.wpscdn.com/avatar/14000253ca4ec9f6853?x-image-process=image/resize,m_fixed,w_180,h_180&k=1783305029341752051",8,"Research & Report","Approximation of Analytic Functions by ReLU Neural Networks with Adjustable Depth and Width","Existing neural network approximation theory often expresses convergence using a single parameter (e.g., parameter count). This work uses an (N, L) characterization—where N is width and L is depth—to study ReLU network approximation for analytic functions, which are infinitely smooth. Upper bounds yield rates of order O(N^{-C L^τ}), with τ determined by the relation between L and N. When N scales roughly as L^d, depth becomes more critical than width, achieved via refined ReLU constructions for power functions, multivariate multiplication, and polynomials.","arXiv :2607 . 10589v1 [ stat .ML] 12 Jul 2026  \nApproximation of Analytic Functions by ReLU Neural Networks with Adjustable Depth and Width  \nYanming Lai∗a, Defeng Sunb , and Yang Wangc  \na,b Department of Applied Mathematics, The Hong Kong Polytechnic  \nUniversity, Hung Hom, Hong Kong, China  \nc Department of Mathematics, The University of Hong Kong, Pokfulam,  \nHong Kong, China  \nAbstract  \nIn contrast to most studies on neural network approximation theory that characterize results through a single parameter, such as the total number of network parameters, [44] pioneered the characterization of approximation rates as a joint function of the width parameter N and the depth parameter L, thereby granting greater architectural flexibility. Existing works using the (N, L)-characterization focus on function classes with finite smoothness s, establishing a typical approximation rate of O 􀀀 N −2s/dL−2s/d􀀁 with d denoting the input dimension, which indicates that network depth and width play symmetric roles for these classes. In contrast, this paper establishes upper bounds for the approximation of analytic functions, which possess infinite smoothness, via ReLU networks under the (N, L)-characterization. Specifically, we derive approximation rates of O 􀀀 N −CLτ􀀁, where C > 0 is some constant and τ > 0 is a parameter influenced by the relation between L and N. In particular, τ = 1 if N scales roughly as Ld. Our findings reveal that depth plays a more critical role than width in the context of analytic function approximation. The main technical difficulty of obtaining such upper bounds lies in the trade-off between the smoothness parameters and the approximation accuracy. To overcome this difficulty, we employ refined constructions of several ReLU networks to approximate power functions, multivariate multiplication, and polynomials, which may be of independent interest.  \n1 Introduction  \n1.1 Background  \nOver the past decade, neural networks have achieved spectacular success across a vast array of practical applications, ranging from computer vision and natural language processing to scientific computing. This empirical triumph is largely attributed to their extraordinary capacity to represent complex, high-dimensional target functions from data.  \n∗ Corresponding Author ([yanming.lai@polyu.edu.hk](yanming.lai@polyu.edu.hk))  \nTo unravel the underlying mathematical mechanisms driving this success, a rigorous analysis of their structural properties is highly demanded. Among various neural network architectures, the foundational model is the feedforward neural network (FNN) . Atypical FNN g : Rd → Rnout is mathematically formulated as:  \ng 0 (x) = x,  \ngℓ+1(x) = σ(Aℓgℓ (x) + bℓ), ℓ = 0 , 1 , . . . , ¯L − 1 ,  \ng (x) = A¯Lg ¯L(x) + b¯L ,  \nwhere Aℓ ∈ Rn ℓ+1×n ℓ , bℓ ∈ Rn ℓ+1 with n0 = d, n¯L+1 = nout denote the weight matrices and bias vectors, respectively. The numbers ¯N := maxℓ∈{1 , ... ,¯L} nℓ and ¯L are called the width and depth of g, respectively. The function σ is called the activation of g, which acts on vectors componentwise. In this work, we will focus on the widely used rectified linear unit (ReLU) activation σR (x) := max{x,0}, which is the simplest piecewise linear function.  \nThe study on the approximation capabilities of FNNs dates back to the late 1980s and 1990s. During that era, research primarily focused on two-layer networks with smooth activation functions. Representative works from this period include [10, 19, 3, 33, 26, 32, 39] and the references therein, with the last being a comprehensive review paper. In the late 2010s, driven by the rapid development of machine learning, the approximation theory of neural networks witnessed a major resurgence of interest. The scope of research has since expanded significantly to encompass both shallow and deep architectures with a diverse array of activation functions, among which ReLU networks have received particular attention. Landmark contributions in this modern wave incl","cbCaihrE89bEf65L","https://ap.wps.com/l/cbCaihrE89bEf65L","pdf",536921,5,1,47,"English","en",105,"# Introduction\n## Background","[{\"question\":\"How does the paper differ from prior neural network approximation theory?\",\"answer\":\"It replaces single-parameter characterizations with an (N, L) view that jointly studies width N and depth L, enabling greater architectural flexibility and clearer separation of their roles.\"},{\"question\":\"What approximation rates does the paper establish for analytic functions?\",\"answer\":\"It derives upper bounds giving approximation rates of the form O(N^{-C L^τ}), where C\\u003e0 is a constant and τ\\u003e0 depends on how L relates to N.\"},{\"question\":\"Why is depth more important than width for analytic function approximation here?\",\"answer\":\"The obtained bounds show a stronger dependence on L than on N for the analytic class under the (N, L) characterization, indicating depth drives the achievable accuracy more critically.\"}]",1784207524,118,{"code":4,"msg":31,"data":32},"ok",{"site_id":25,"language":24,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":87,"head_meta":89,"extra_data":91,"updated_unix":28},"approximation-of-analytic-functions-by-relu-neural-networks-with-adjustable-depth-and-width","",{"@graph":36,"@context":86},[37,54,69],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/approximation-of-analytic-functions-by-relu-neural-networks-with-adjustable-depth-and-width/85977/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":24,"description":14,"dateModified":62,"datePublished":63,"encodingFormat":61,"isAccessibleForFree":64,"interactionStatistic":65},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-07-26","2026-07-16",true,{"@type":66,"interactionType":67,"userInteractionCount":20},"InteractionCounter",{"@type":68},"ViewAction",{"@type":70,"mainEntity":71},"FAQPage",[72,78,82],{"name":73,"@type":74,"acceptedAnswer":75},"How does the paper differ from prior neural network approximation theory?","Question",{"text":76,"@type":77},"It replaces single-parameter characterizations with an (N, L) view that jointly studies width N and depth L, enabling greater architectural flexibility and clearer separation of their roles.","Answer",{"name":79,"@type":74,"acceptedAnswer":80},"What approximation rates does the paper establish for analytic functions?",{"text":81,"@type":77},"It derives upper bounds giving approximation rates of the form O(N^{-C L^τ}), where C>0 is a constant and τ>0 depends on how L relates to N.",{"name":83,"@type":74,"acceptedAnswer":84},"Why is depth more important than width for analytic function approximation here?",{"text":85,"@type":77},"The obtained bounds show a stronger dependence on L than on N for the analytic class under the (N, L) characterization, indicating depth drives the achievable accuracy more critically.","https://schema.org",{"og:url":52,"og:type":88,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":90,"canonical":52},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":93},[94,98,102,106,110,115,120,123,128,131,135],{"id":21,"doc_module":4,"doc_module_name":46,"category_name":95,"show_sort_weight":96,"slug":97},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":99,"show_sort_weight":100,"slug":101},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":103,"show_sort_weight":104,"slug":105},"Exam",70,"exam",{"id":20,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":20,"slug":138},19,"General","general"]