Deep Reinforcement Learning for On-demand Intelligent Routing in Deterministic Networks

Deterministic networks have an obligation to guarantee deterministic transmission requirements of various applications in terms of delay, packet loss, throughput, and reliability. Traditional routing mechanisms, however, do not take sufficient advantage of the abundant network resources and can only provide limited quality of service (QoS) guarantees. Therefore, we propose an on-demand intelligent routing (OdIR) framework for deterministic networks. First, built on in-band network telemetry (INT) implemented by programming protocol-independent packet processors (P4), we design a fine-grained and high-precision awareness strategy to obtain network state information in real-time. Second, we present an intelligent routing decision approach based on the improved deep deterministic policy gradient (DDPG) algorithm. Finally, we adopt the segment routing MPLS (SR-MPLS) paradigm in the data plane to forward deterministic flows according to decision paths. Simulation results show that the OdIR framework can effectively reduce link overhead, end-to-end delay, packet loss rate, and southbound communication overhead under guaranteed deterministic QoS compared with traditional routing mechanisms.