Relevance Vector Sampling for Reinforcement Learning in Continuous Action Space

To be applicable to real world problems, much reinforcement learning (RL) research has focused on continuous state spaces with function approximations. Some problems also require continuous actions, but searching for good actions in a continuous action space is problematic. This paper suggests a novel relevance vector sampling approach to action search in an RL framework with relevance vector machines (RVM-RL). We hypothesize that each relevance vector (RV) is placed on the modes of the value approximation surface as the learning converges. From the hypothesis, we select actions in RVs to maximize the estimated state-action values. We report the efficiency of the proposed approach by controlling a simulated octopus arm with RV-sampled actions.