A deep reinforcement learning hyper-heuristic with feature fusion for online packing problems

In recent years, deep reinforcement learning has shown great potential in solving computer games with sequential decision-making scenarios. Hyper-heuristic is a generic search framework, capable of intelligently selecting or generating algorithms to solve a class of optimisation problems with stochastic or dynamic settings. This paper proposes a new general framework for solving online packing problems using deep reinforcement learning hyper-heuristics. Although analytical approaches can address most offline packing problems successfully, their online versions have proved much more challenging and the performance of the existing methods is often not satisfactory. In this paper, we extend a recent deep reinforcement learning hyper-heuristic framework by fusing the visual information of real-time packing with distributional information of random parameters of the problem. Computational experiments show that our method outperforms the state of the art online methods with reductions in optimality gap between 2%-19% for knapsack problem and 0.7% for the online strip packing problem. In addition, a new visual analysis presentation is also devised to better interpret the learned packing strategies, which can reveal more information than the widely used landscape analysis. As online packing problems are widely available in production environments, the proposed approach can serve as an important reference to solve other similar combinatorial optimisation problems for which visual layout inputs would aid learning.