Hierarchical Correlations Replay for Continual Learning

Continual Learning (CL) aims at incrementally learning new knowledge from an infinite stream of data while preserving old knowledge. A perfect CL method is expected to retain the old knowledge greedily to alleviate the catastrophic interference caused by the new knowledge. However, most existing methods only focus on exploring the knowledge carried by the instance itself but neglect the correlations of inter-instance and inter-class, which are also valuable for preserving knowledge. To this end, we propose a novel method dubbed Hierarchical Correlations Replay (HCR) consisting of an Instance-level Correlation Replay (ICR) module and a Class-level Correlation Replay (CCR) module, which retain both the instance-level and the class-level correlations to consolidate old knowledge. Specifically, the ICR module employs a correlation matrix to represent the instance-level correlation, and a random triplet probability is utilized to construct the class-level correlation in the CCR module. Extensive experiments on five benchmark image datasets show that our HCR is competitive with or superior to state-of-the-art methods under diverse continual learning settings. (C) 2022 Elsevier B.V. All rights reserved.