Two-armed bandit strategies that discount past and future

We explore the small-sample performance of m-step look ahead strategies that only use the k most recent observations, for the Bernoulli two-armed bandit problem. The larger k is not always the better. Strategies with small k and m perform almost as well as the optimal strategies, and they dominate the optimal one over a non-negligible part of the parameter space. Given that they adapt better to unplanned non-stationarities and outperform the optimal one under small prior misspecifications, they will be hard to beat in applications.