Cooperative behavior of agents that model the other and the self in noisy iterated prisoners' dilemma simulation

We developed self-learning for simulation study of mutual understanding between peer agents. We designed them to use various types of co-player models and a reinforcement-learning algorithm to learn to play a noisy iterated Prisoners' Dilemma game so that the pay-off for the agent itself is maximized. We measured the mutual-modeling ability of each type of agent in terms of cooperative behavior when playing with another equivalent agent. We observed that agents with a complex co-player model, which includes a model of the agent itself, showed higher cooperation than agents with a simpler co-player model only. Moreover, in low-noise environments, Level-M agent, which develops equivalent models of the self and the other, showed higher cooperation than other types of agents. These results suggest the importance of "self-observation" in the design of communicative agents.