When to Elicit Preferences in Multi-Objective Bayesian Optimization

We consider the case of interactive multi-objective Bayesian optimization where decision maker (DM) preferences can be elicited by asking the DM to select the more preferred among pairs of observations. Assuming that there is a cost to evaluating a solution as well as to eliciting preferences, and given a total budget, we propose an acquisition function that, in each iteration, decides whether to evaluate another solution or to query the DM. Thus, the approach automatically chooses how often and when to interact with the DM. It furthermore decides which pair of observations is likely to be most informative when shown to the DM. We show empirically that the proposed criterion is not only able to pick suitable pairs of observations, but also automatically results in a sensible balance between optimization and querying the DM.