Deep Reinforcement Learning for AoI Aware VNF Placement in Multiple Source Systems

Age of Information (AoI) is a newly emergent performance metric to quantify the freshness of data from destinations' perspectives. In this paper, we investigate and analyze AoI in the context of a multiple source updating system. In such a system, multiple IoT devices continuously monitors physical environment and sends data to a remote destination for status updates through a Network Function Virtualization (NFV)-enabled network. Considering that the Virtual Network Function (VNF) placement can unnecessarily influence the AoI of the updates, we study the VNF placement problem in such a system. The problem is hence formulated as a mathematical optimization problem aiming to minimize the long-term average AoI of all updates received at the destination. To solve this problem, we propose a Deep Reinforcement Learning (DRL)-based VNF placement approach called VNF-AoI, where a learning agent or decision-maker interacts with a system environment and consequently provides an optimal VNF placement policy according to the experience it has learned. Finally, we conduct extensive simulations to validate the effectiveness of our proposed approach. Numerical results clearly demonstrate that our VNFAoI surpasses other two baseline algorithms by averagely 13.8% higher acceptance ratio and 20.3% lower average AoI at the destination.