DQR: An Efficient Deep Q-Based Routing Approach in Multi-Controller Software Defined WAN (SD-WAN)

Software Defined Networking (SDN) is a promising paradigm in the field of network technology. This paradigm suggests the separation between the control plane and the data plane which brings flexibility, efficiency and programmability to network resources. SDN deployment in large scale networks raises many issues which can be overcame using a collaborative multi-controller approaches. Such approaches can resolve problems of routing optimization and network scalability. In large scale networks, such as SD-WAN, routing optimization consists of achieving a trade-off between per-flow QoS, the load balancing in each domain as well as the resource utilization in inter-domain links. Multi-Agent Reinforcement Learning paradigm(MARL) is one of the most popular solutions that can be used to optimize routing strategies in SD-WAN. This paper proposes an efficient approach based on MARL which is able to ensure a load balancing among each network as well as optimized resource utilization of inter-domain links. This approach profits from our previous work, denoted SPFLR, and tries to balance the load of the whole network using Deep Q-Networks (DQN) algorithms. Simulation results show that the proposed solution performs better than parallel solutions such as BGP-based routing and random routing.