Dynamic Joint VNF Forwarding Graph Composition and Embedding: A Deep Reinforcement Learning Framework

Network Function Virtualization (NFV) is a network service deployment technology that reduces capital and operational costs while yielding flexibility and scalability for service operators. As such, an ordered chain of Virtual Network Functions (VNFs), known as a VNF Forwarding Graph (VNF-FG), should be composed and embedded into the underlying substrate network. In the literature, the composition and embedding stages of VNF-FGs are usually targeted separately, which may result in undesired solutions. In this paper, we propose our joint VNF-FG composition and embedding solution, which considers the variations of service demands while also accounting for dynamic network conditions. Specifically, our proposed solution relies on deep reinforcement learning empowered by two components for estimating dynamic parameters: network resource utilization and service demand analyzers. Moreover, to efficiently explore the problem's large discrete action space, we utilize a specialized branching Q-network and enhance it with an action filtering mechanism. We evaluated our proposed method against joint and disjoint composition and embedding heuristics as well as versus other deep learning-based methods. Our results show that the proposed method can achieve up to a 95% improvement of embedding cost compared to our benchmarks.