A margin-based multiclass generalization bound via geometric complexity

There has been considerable effort to better understand the generalization capabilities of deep neural networks both as a means to unlock a theoretical understanding of their success as well as providing directions for further improvements. In this paper we investigate margin-based multi-class generalization bounds for neural networks which rely on a recent complexity measure, the geometric complexity, developed for neural networks and which measures the variability of the model function (Dherin et al., 2022). We derive a new upper bound on the generalization error which scales with the marginnormalized geometric complexity of the network and which holds for a broad family of data distributions and model classes. Our generalization bound is empirically investigated for a ResNet18 model trained with SGD on the CIFAR10 and CIFAR100 datasets with both original and random labels.