Virtual Network Embedding Based on Hierarchical Cooperative Multiagent Reinforcement Learning

Virtual network embedding (VNE) is a promising technique enabling 5G networks to satisfy the given requirements of each service via network virtualization (NV). For better performance of the embedding algorithm, it is necessary to automatically detect the network status and provide an optimal embedding decision. However, existing virtual network embedding (VNE) algorithms disregard the long-term effect by focusing on selecting only one virtual network request (VNR) from the waiting queue, without considering all waiting virtual network requests concurrently. In this study, we propose a hierarchical cooperative multiagent reinforcement learning (MARL) algorithm to optimize the VNE problem by maximizing average revenue, minimizing average cost, and also improving the request acceptance ratio. The proposed algorithm applies two acrshort RL algorithms: 1) two-level hierarchical RL (HRL) to efficiently solve the problem by dividing it into subproblems and 2) multiagent-based cooperative acrshort RL to improve algorithm performance through the cooperation of multiple agents. In order to evaluate and analyze the proposed scheme from the long-term perspective, four performance parameters are evaluated: 1) revenue; 2) cost; 3) revenue-to-cost ratio; and 4) acceptance ratio. The simulation results demonstrate that the proposed VNE algorithm based on hierarchical and acrshort MARL outperforms the existing acrshort RL-based approaches.