Ad Hoc Teamwork in the Presence of Non-stationary Teammates

In this paper we address the problem of ad hoc teamwork and contribute a novel approach, PPAS, that is able to handle non-stationary teammates. Current approaches to ad hoc teamwork assume that the (potentially unknown) teammates behave in a stationary way, which is a significant limitation in real world conditions, since humans and other intelligent systems do not necessarily follow strict policies. In our work we highlight the current limitations of state-of-the-art approaches to ad hoc teamwork problem in the presence of non-stationary teammate, and propose a novel solution that alleviates the stationarity assumption by combining ad hoc teamwork with adversarial online prediction. The proposed architecture is called PLASTIC Policy with Adversarial Selection, or PPAS. We showcase the effectiveness of our approach through an empirical evaluation in the half-field offense environment. Our results show that it is possible to cooperate in an ad hoc manner with non-stationary teammates in complex environments.