A grammar-based GP approach applied to the design of deep neural networks

Deep Learning has been very successful in automating the feature engineering process, widely applied for various tasks, such as speech recognition, classification, segmentation of images, time-series forecasting, among others. Deep neural networks (DNNs) incorporate the power to learn patterns through data, following an end-to-end fashion and expand the applicability in real world problems, since less pre-processing is necessary. With the fast growth in both scale and complexity, a new challenge has emerged regarding the design and configuration of DNNs. In this work, we present a study on applying an evolutionary grammar-based genetic programming algorithm (GP) as a unified approach to the design of DNNs. Evolutionary approaches have been growing in popularity for this subject as Neuroevolution is studied more. We validate our approach in three different applications: the design of Convolutional Neural Networks for image classification, Graph Neural Networks for text classification, and U-Nets for image segmentation. The results show that evolutionary grammar-based GP can efficiently generate different DNN architectures, adapted to each problem, employing choices that differ from what is usually seen in networks designed by hand. This approach has shown a lot of promise regarding the design of architectures, reaching competitive results with their counterparts.