Incorporating Transfer Learning in CNN Architecture

Machine learning (ML) is a data intensive process. For training of ML algorithms huge datasets are required. There are times when enough data is not available due to multitude of reasons. This could be due to lack of availability of annotated data in a particular domain or paucity of time in data collection process resulting in non-availability of enough data. Many a times data collection is very expensive and in few domains data collection is very difficult. In such cases, if methods can be designed to reuse the knowledge gained in one domain having enough training data, to some other related domain having less training data, then problems associated with lack of data can be overcome. Transfer Learning (TL) is one such method. TL improves the performance of the target domain through knowledge transfer from some different but related source domain. This knowledge transfer can be in form of feature extraction, domain adaptation, rule extraction for advice and so on. TL also works with various kinds of ML tasks related to supervised, unsupervised and reinforcement learning. The Convolutional Neural Networks are well suited for the TL approach. The general features learned on a base network (source) are shifted to the target network. The target network then uses its own data and learns new features specific to its requirement.