A Machine Learning Approach for Service Function Chain Embedding in Cloud Datacenter Networks

Network Functions Virtualization (NFV) is an industry effort to replace traditional hardware middleboxes with virtualized network functions (VNFs) running on general-build hardware platforms, enabling cost reduction, operational efficiency, and service agility. A Service Function Chain (SFC) constitutes an end-to-end network service, formed by chaining together VNFs in specific order. Infrastructure providers and cloud service providers try to optimally allocate computing and network resources to SFCs, in order to reduce costs and increase profit margins. The corresponding resource allocation problem, known as SFC embedding problem, is proven to be NP-hard. Traditionally the problem has been formulated as Mixed Integer Linear Program (MILP), assuming each SFC's requirements are known a priori, while the embedding decision is based on a snapshot of the infrastructure's load at request time. Reinforcement learning (RL) has been recently applied, showing promising results, specifically in dynamic environments, where such assumptions are considered unrealistic. However, standard RL techniques such as Q-learning might not be appropriate for addressing the problem at scale, as they are often ineffective for high-dimensional domains. On the other hand, Deep RL (DRL) algorithms can deal with high dimensional state spaces. In this paper, a Deep Q-Learning (DQL) approach is proposed to address the SFC resource allocation problem. The DQL agent utilizes a neural network for function approximation in Q-learning with experience replay learning. The simulations demonstrate that the new approach outperforms the linear programming approach. In addition, the DQL agent can perform SFC request admission control in real time.