A Modified Average Reward Reinforcement Learning Based on Fuzzy Reward Function

The purpose of this paper is to propose a fuzzy reward function for improving the learning efficiency of reinforcement learning. Reinforcement learning is a sort of on-line learning approach. During the learning process, learning system (often called an agent) learns how to operate in the environment, basing upon the effect of action-reward signal. Actually, the reward signal is always represented with a reward function, of which the role is to evaluate whether the agent acts well or not. According to the distribution of rewards in the space of states, reward function can be classified as two basic forms, dense function and sparse function. Parse function has the advantage of the easy implementation, but its learning efficiency is relatively low. As for dense function, it is very difficult to design when the size of state space is very large. In response to these defects, we present a methodology for designing a fuzzy reward functions by the use of fuzzy inference system. As a result of this solution, not only generalization ability of reward function can be improved, but also we are apt to embed expert experience in reinforcement learning system. Through applying the fuzzy reward function to R-learning and making simulation experiment, we verify that the learning efficiency for R-learning is effectively improved.