Service Function Chain Embedding Meets Machine Learning: Deep Reinforcement Learning Approach

With the emerge of the network function virtualization (NFV) and software-defined network (SDN), the SDN/NFV-enabled network has been recognized as one of the most promising technologies to efficiently achieve resource allocation for network service. By introducing the SDN/NFV technology, each service can be represented by a service function chain (SFC), which can deploy the virtualized network functions (VNFs) and chain them with corresponding flows allocation. Considering the dynamic and complex nature of mobile terminals in cloud networks, how to efficiently embedding SFCs remains as a challenging problem. However, the traditional methods (e.g., exact, heuristic, meta-heuristic, and game, etc.) are subjected to the complexity of cloud network scenarios with dynamic network states, high-speed computational requirements, and enormous service requests. Recent studies have shown that deep reinforcement learning (DRL) is a promising way to deal with the limitations of the traditional methods. However, DRL agent training easily suffers from the problem of slow convergence performance. In order to overcome this narrow, in this paper, we design a novel DRL framework based on the enhanced deep deterministic policy gradient (E-DDPG) for the efficient SFC embedding in the dynamic and complex cloud network scenarios. Simulation results validate the high efficiency of the proposed DRL framework as it not only converges faster than currently baseline algorithms, but also reduces the end-to-end delay down to at least 28.3% compared to the benchmarks. All our proposed algorithms and code are available at https://github.com/jn-z/.