Guiding the Evolution of Genetic Network Programming with Reinforcement Learning

Genetic Network Programming (GNP) is one of the evolutionary algorithms. It adopts a directed graph structure to represent a solution to a given problem. Agents judge situations and execute actions sequentially following the node transitions in the graph. On one hand, GNP possesses an advantage of node reusability, which makes it possible to realize a compact graph structure that represents a solution. On the other hand, the compact structure suggests that any connection might play a significant role in the solution, i.e., a slight change to the connections could tremendously influence the performance of the agents for the given task. The conventional GNP, however, lacks an effective way to evaluate and to take advantage of the connections. This paper thus proposes a reinforcement learning approach to learn GNP's subgraphs that contain a relatively small number of connections, and further proposes a partial reconstruction approach to modify the solution with the obtained subgraphs. These two approaches are combined together to form a new evolutionary learning model named GNP with Evolution-oriented Reinforcement Learning (GNP-ERL). Some experiments are conducted on the Tileworld testbed to verify the effectiveness of GNP-ERL, and the simulation results demonstrate that it outperforms the conventional GNP in both training and testing phases.