Coordinating Experience Replay: A Harmonious Experience Retention approach for Continual Learning

Continual Learning (CL) aims at learning new knowledge gradually from an infinite data streams while retaining the old knowledge. A key problem in CL is the stability-plasticity dilemma, i.e., a high stability weakens the model from learning new data, while a high plasticity results in forgetting old classes. To alleviate this dilemma, some work has been reported to replay a finite subset of stored examples when training on new tasks, while the effectiveness heavily depends on the rehearsal mode and the validity of these exemplars. To this end, this paper proposes a Coordinating Experience Replay approach consisting of a Harmonious Experience Retention approach and a Vital Exemplar Reserved Sampling strategy, which constrains the rehearsal process and develops two criteria to select examples for better replaying the beneficial experience. Specifically, for the Harmonious Experience Retention approach, an indirect rehearsal process is supplemented on Dark Experience Replay by Imitative Experience Replay (IER), which imitates the response process via constructing empirical constraints on the saved logits and ground-truth labels directly. For the Vital Exemplar Reserved Sampling strategy, we apply the classification loss value and the class information as the criteria to select examples, which balance the criteria of class integrity and example vitality. Extensive experiments on seven benchmark datasets show the superiority of the proposed approach under diverse continual learning settings. (C) 2021 Elsevier B.V. All rights reserved.