ML-based Arm Recommendation in Short-Horizon MABs

In many settings where an agent needs to suggest or recommend a course of action to its user, the agent's goal may not fully align with the user's goal. In particular, the agent may maximize its benefit if the user chooses specific alternatives that are not necessarily the ones that maximize her own individual benefit. In this paper we study such setting in the context of providing advice in two-armed bandit problems. We explore a potential strategy for the agent aiming to influence the arm to be picked. In particular we focus on a somehow naive recommendation strategy that always recommend the preferred arm and a strategy that recommends based on various Machine Learning models that aim to guide the decision regarding when to switch to the agent's least preferred arm. Based on extensive evaluation we find that both recommendation strategies results in better performance compared to not making any recommendation, and that the naive recommendation strategy performs slightly better than the ML-based recommendations, despite using a substantial amount of training data for the latter.