Availability-aware and energy-aware dynamic SFC placement using reinforcement learning

Software-defined networking and network functions virtualisation are making networks programmable and consequently much more flexible and agile. To meet service-level agreements, achieve greater utilisation of legacy networks, faster service deployment, and reduce expenditure, telecommunications operators are deploying increasingly complex service function chains (SFCs). Notwithstanding the benefits of SFCs, increasing heterogeneity and dynamism from the cloud to the edge introduces significant SFC placement challenges, not least adding or removing network functions while maintaining availability, quality of service, and minimising cost. In this paper, an availability- and energy-aware solution based on reinforcement learning (RL) is proposed for dynamic SFC placement. Two policy-aware RL algorithms, Advantage Actor-Critic (A2C) and Proximal Policy Optimisation (PPO), are compared using simulations of a ground truth network topology based on the Rede Nacional de Ensino e Pesquisa Network, Brazil’s National Teaching and Research Network backbone. The simulation results show that PPO generally outperformed A2C and a greedy approach in terms of both acceptance rate and energy consumption. The biggest difference in the PPO when compared to the other algorithms relates to the SFC availability requirement of 99.965%; the PPO algorithm median acceptance rate is 67.34% better than the A2C algorithm. A2C outperforms PPO only in the scenario where network servers had a greater number of computing resources. In this case, the A2C is 1% better than the PPO.