A Deep Autoencoder-Based Knowledge Transfer Approach

Deep Transfer Learning or DTS has proven successful with deep neural networks and deep belief networks. However, there has been limited research on to using deep autoencoder (DAE)-based network to implement DTS. This paper for the first time attempts to identify transferable features in the form of learning and transfer them to another network implementing a simple DTS mechanism. In this paper, a transfer of knowledge process is proposed where in knowledge is transferred from one Deep autoencoder network to another. This knowledge transfer has helped to improve the classification accuracy of the receiving autoencoder, particularly when experimented using corrupted dataset. The experiments are carried out on a texa based hierarchical dataset. Firstly, a DAE is trained with regular undamaged dataset to achieve maximum accuracy. Then, a distorted dataset was used to train second DAEN for classification with which only 56.7% of the data is correctly classified. Then a set of weights are transferred from from first DAEN to the second DAEN which resulted in an an improvement of classification accuracy by about 22%. The key contribution of this paper is highlighting importance of knowledge transfer between two deep autoencoder networks which is proposed for the first time.