Minimum Block Width for Universal Approximation by Residual Neural Networks with Inner Width One

Q: What is the main goal regarding residual neural networks in this paper?

The paper analyzes the universal approximation property of ResNets and establishes new results on the minimum block width required for approximation.

Q: When each residual branch has inner width 1, what is the exact minimum block width for Lp approximation?

For Lp approximation on any compact domain (1 ≤ p < ∞), the exact minimum block width is max{dx, dy}.

Q: Does the paper guarantee that smaller block width than max{dx, dy} can approximate all target functions?

No. It proves that for any activation function family there exist functions that cannot be approximated in either the Lp sense or the uniform sense when the block width is less than max{dx, dy}, regardless of inner width.

Jul 16, 2026

Upload by Eliana

This paper studies the universal approximation property of residual neural networks (ResNets) and derives new sharp bounds on the minimum block width. For input/output dimensions dx and dy, and activations including LeakyReLU, ReLU, and ReLU-like functions, the work establishes upper and lower bounds. Under inner width 1, the exact minimum block width for Lp approximation on any compact domain is max{dx,dy}, and a compatible range of block widths yields uniform approximation. For any activation family, it proves existence of target functions requiring block width at least max{dx,dy} even irrespective of inner width.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages34

Reading Time1 h 26 min

Abstract

Keywords

Introduction

Motivation

Related Work

arXiv :2607 .04597v2 [ cs .LG] 13 Jul 2026 Minimum Block Width for Universal Approximation by Residual Neural Networks with Inner Width One Qi Zhou 1, 2 , Xuan Zhou 1, 2 , Xiao-Song Yang 1, 2* 1 School of Mathematics and Statistics, Huazhong University of Science and Technology, Wuhan, 430074, Hubei, P.R. China. 2 Hubei Key Laboratory of Engineering Modeling and Scientific Computing, Huazhong University of Science and Technology, Wuhan, 430074, Hubei, P.R. China. *Corresponding author(s). E-mail(s): [yangxs@hust.edu.cn](yangxs@hust.edu.cn) ; Contributing authors: [qizhou1037@hust.edu.cn](qizhou1037@hust.edu.cn) ; [xuanzhou1037@hust.edu.cn](xuanzhou1037@hust.edu.cn) ; Abstract In this paper, we study the universal approximation property of residual neural networks, and obtain some new results. For input and output dimensions dx and dy , and LeakyReLU, ReLU, ReLU-like activation functions, the upper and lower bounds of the minimum block width are established. To achieve Lp approximation (1 ≤ p < +∞) on any compact domain, we show that the exact minimum block width is max{dx , dy } when each residual branch has inner width 1 . Furthermore, we show that residual neural networks with block width min{dx + dy , max{2dx + 1, dy }} can achieve uniform approximation on any compact domain under the constraint that each residual branch has inner width 1. Besides, for any activation function family, we prove that there exist functions that cannot be approximated by residual neural networks with block width less than max{dx , dy }, both in the Lp sense and the uniform sense, regardless of inner width. 1 Keywords: Minimum Width, Residual Neural Networks, Compact Uniform Approximation, Universal Approximation Property MSC Classification: 41A46 , 41A63 , 41A65 , 68T07 1 Introduction 1.1 Motivation Residual Neural Networks (ResNets) play an important role in machine learning tasks (e.g., image recognition [8], object detection [9]) . Thus, the approximation theory of ResNets has become an important topic in the mathematical theory of deep learning. Mathematically, a ResNet can be viewed as a composition of residual blocks. Each residual block consists of an identity map and a residual branch, where the residual branch is itself a small neural network. The outputs of these two components are combined by vector addition. Thus, each residual block acts on Rw according to B (x) = x + Φ(x), where the identity map provides the shortcut connection and Φ denotes the residual branch. In particular, ResNets have been interpreted as time discretizations of ordinary differential equation (ODE) flows on Rdx [5], and it has been proved that ResNets can approximate every function which can be connected to an identity function through an analytic and monotone homotopy [23] . However, recent studies have indicated that flow-based residual transformations may be limited by topological obstructions of the input set [4] . As a result, ODE-induced flows may fail to approximate arbitrary continuous maps from Rdx to itself because of topological obstructions. This motivates the augmented ResNet architecture, which first lifts the input from Rdx into a higher dimensional ambient space Rw by an initial linear layer, then performs residual transformations in the enlarged space, and finally projects the transformed input set from Rw onto the desired output space Rdy through the final linear layer. Due to the benefits of augmented ResNets [27], we adopt this augmented ResNet architecture throughout this paper. In contrast to the width notion for Multi-Layer Perceptrons (MLPs), augmented ResNets have two width parameters. On the one hand, each residual block acts on the ambient space Rw , then we call this dimension the block width w. On the other hand, the residual branch inside each block is itself a small neural network, and the width of the residual branch is called the inner width w¯ (see Definition 8) . These two quantities m

FAQs

What is the main goal regarding residual neural networks in this paper?

The paper analyzes the universal approximation property of ResNets and establishes new results on the minimum block width required for approximation.

When each residual branch has inner width 1, what is the exact minimum block width for Lp approximation?

For Lp approximation on any compact domain (1 ≤ p < ∞), the exact minimum block width is max{dx, dy}.

Does the paper guarantee that smaller block width than max{dx, dy} can approximate all target functions?

No. It proves that for any activation function family there exist functions that cannot be approximated in either the Lp sense or the uniform sense when the block width is less than max{dx, dy}, regardless of inner width.

Jul 16, 2026

Upload by Eliana