Learning to Balance Exploration and Exploitation in Pareto Local Search for Multi-objective Combinatorial Optimization

As a natural extension of local search, Pareto local search (PLS) is a basic building block in many state-of-the-art metaheuristics for multi-objective combinatorial optimization problems (MCOPs). However, the basic PLS suffers from a low convergence rate, since it always fully explores the neighborhood of each unexplored solution, which is unnecessary. Some studies tried to introduce heuristic design in PLS to balance exploration and exploitation. In this paper, we handle this issue by a learning based framework. In the framework, PLS applies the firstK strategy, namely it stops exploring a solution's neighborhood when it obtains K non-dominated solutions, where K is adaptively controlled by a neural network based on observations collected during the search. Training the neural network is modeled as a reinforcement learning problem. Thus the proposed PLS variant is called PLSNN. In the experiments, we compared the performance of PLSNN and several PLS variants with heuristic design on the multi-objective unconstrained binary quadratic programming problem (mUBQP). The experimental results show that PLSNN performed significantly better than its counterparts on all test instances.