A General-Purpose Neural Architecture Search Algorithm for Building Deep Neural Networks

With the increasing availability of data and the development of powerful algorithms, deep neural networks have become an essential tool for all sectors. However, it can be challenging to automate the process of building and tuning them, due to the rapid growth of data and their complexity. The demand for handling large amounts of data has led to an increasing number of hidden layers and hyperparameters. A framework or methodology to design the architecture of deep neural networks will be crucial in the future, as it could significantly speed up the process of using deep learning models. We present here a first attempt to create an algorithm that combines aspects of Neural Architecture Search and Hyperparameter Optimization to build and optimize a neural network architecture. The particularity of our algorithm is that it is able to learn how to link neural layers of different types to create increasingly performant neural network architectures. We conducted experiments on four different tasks, including regression, binary and multi-classification, and forecasting, to compare our algorithm with common machine learning models.