Latent Context Based Soft Actor-Critic

The performance of deep reinforcement learning methods prone to degenerate when applied to tasks requiring relatively longer horizon memory or with highly variable dynamics. In this paper, we utilize the probabilistic latent context variables motivated by recent Meta-RL materials, and propose the Latent Context based Soft Actor-Critic (LC-SAC) approach to address aforementioned issues. The latent context is capable to encode information about both the agent's previous behaviors and the dynamics of the current undergoing environment, which empirically believed to be beneficial for efficient policy optimization. Experiment results demonstrate that LC-SAC can achieve comparable performance with SAC on a collection of continuous control benchmarks and outperforms SAC in some particular tasks with above two characteristics. Moreover, we also introduce a simple but general procedure to integrate LC-SAC with diversequality demonstrations to enable efficient reuse of human prior knowledge, and finally achieve competitive performance with comparatively small number of interactions with environments.