A cloud-fog computing framework for real-time energy management in multi-microgrid system utilizing deep reinforcement learning

Uncertainties in a microgrid (MG) result in challenges in reaching the optimal production-consumption balance via the energy management system (EMS). Therefore, multi-MG systems are proposed to achieve more optimality with stability. However, the EMS yet faces more uncertainties due to the increased number of renewable energy resources in multi-microgrids. Therefore, the use of a battery energy storage system (BESS) is crucial to manage these uncertainties. BESSs impose huge investment and operation costs, so it is important to consider their optimal planning and operation to maximize their benefits and lifespan. Model-based optimization approaches are used by formulating the EMS problem based on the complete system models under uncertainties. However, this assumption is usually impractical due to the prohibitive complexity and computational burden of solving a large nonlinear problem with many uncertain variables subject to privacy policies. This paper employs the deep reinforcement learning (DRL) technique to handle uncertainties associated with the large number of uncertain variables in EMS for multi-MG systems. An auxiliary cloud-fog computing framework is proposed for the DRL agents, which includes sufficient storage space, computational resources, and communication infrastructure among MGs. Simulation results in Matlab reveal that the optimality of the EMS is improved by 15 % on average by utilizing the auxiliary computing framework.