Deep ReLU neural networks in high-dimensional approximation

We study the computation complexity of deep ReLU (Rectified Linear Unit) neural networks for the approximation of functions from the Holder-Zygmund space of mixed smoothness defined on the d-dimensional unit cube when the dimension d may be very large. The approximation error is measured in the norm of isotropic Sobolev space. For every function f from the Holder-Zygmund space of mixed smoothness, we explicitly construct a deep ReLU neural network having an output that approximates f with a prescribed accuracy epsilon, and prove tight dimension-dependent upper and lower bounds of the computation complexity of the approximation, characterized as the size and depth of this deep ReLU neural network, explicitly in d and epsilon. The proof of these results in particular, relies on the approximation by sparse-grid sampling recovery based on the Faber series. (C) 2021 Elsevier Ltd. All rights reserved.