DATA-EFFICIENT DEEP REINFORCEMENT LEARNING WITH CONVOLUTION-BASED STATE ENCODER NETWORKS

Due to its ability to deal with high-dimensional end-to-end learning control problems, deep reinforcement learning (DRL) has received lots of research interests in recent years. However, the existing DRL approaches still face the challenge of data efficiency and the online learning control performance of DRL algorithms still needs to be improved. In this paper, we propose an online DRL approach with convolutional encoder networks. In the proposed approach, a cascaded learning control architecture is designed, which performs system state extraction and dimension reduction in the first stage and executes online reinforcement learning in the second stage. A convolutional network is used to encode features from the raw image data so that the algorithm can be implemented based on the encoded low-dimensional features, which can significantly improve the learning efficiency. Experimental results on two benchmark of learning control tasks show that the proposed approach outperforms previous end-to-end DRL approaches, which demonstrates the effectiveness and efficiency of the proposed approach.