Continual Learning with Bayesian Model Based on a Fixed Pre-trained Feature Extractor

Current deep learning models are characterised by catastrophic forgetting of old knowledge when learning new classes. This poses a challenge in intelligent diagnosis systems where initially only training data of a limited number of diseases are available. In this case, updating the intelligent system with data of new diseases would inevitably downgrade its performance on previously learned diseases. Inspired by the process of learning new knowledge in human brains, we propose a Bayesian generative model for continual learning built on a fixed pretrained feature extractor. In this model, knowledge of each old class can be compactly represented by a collection of statistical distributions, e.g. with Gaussian mixture models, and naturally kept from forgetting in continual learning. Experiments on two skin image sets showed that the proposed approach outperforms state-of-the-art approaches which even keep some images of old classes during continual learning of new classes.