Improved interpretable method with experimental verification based on adaptive temperature

被引：0
|
作者：
Ben K. [1 ]
Wang T. [1 ]
Zhang X. [1 ]
机构：
[1] College of Electronic Engineering, Navy University of Engineering, Wuhan
来源：
Huazhong Keji Daxue Xuebao (Ziran Kexue Ban)/Journal of Huazhong University of Science and Technology (Natural Science Edition) | 2022年 / 50卷 / 02期
关键词：
Adaptive temperature; Ex-post interpretable method; Interpretability; Model distillation; Small teacher network;
D O I：
10.13245/j.hust.220219
中图分类号：
学科分类号：
摘要：
Aiming at the low efficiency problem and high time cost of hyperparameter temperature in teacher-student network, an interpretable model based on adaptive temperature assisted training of the small teacher network was proposed. On the basis of the original teacher-student model structure, firstly, it shows that the temperature hyperparameter is only related to the training convergence speed of the student model. Secondly, the small teacher model structure was added to save the training time of the interpretation model. In the verification experiment of image classification, the accuracy of the interpretation model in cifar-100 is increased by 2.45% compared with the original model, and the processing time is saved by 26.33%. The proposed method can make a global approximation to the interpretation model, and it is an ex-post interpretable method with a short processing time. © 2022, Editorial Board of Journal of Huazhong University of Science and Technology. All right reserved.
引用
收藏
页码：124 / 129
页数：5
相关论文
共 12 条
  • [1] 6
  • [2] 29, pp. 98-105, (2017)
  • [3] YU Z, LIU F, LIAO R, Et al., Improvement of face recognition algorithm based on neural network, Proc of the 10th International Conference on Measuring Technology and Mechatronics Automation, pp. 229-234, (2018)
  • [4] 45, (2018)
  • [5] HINTON G, VINYALS O, DEAN J., Distilling the knowledge in a neural network, Computer Science, 14, 7, pp. 38-39, (2015)
  • [6] HOWARD A, ZHU M, CHEN B, Et al., MobileNets: efficient convolutional neural networks for mobile vision applications, Proc of Computer Vision Papers, pp. 1462-1471, (2017)
  • [7] HA J, MOON H, KWAK J, Et al., Deep convolutional neural network for classifying fusarium wilt of radish from unmanned aerial vehicles, Journal of Applied Remote Sensing, 11, 4, pp. 621-632, (2017)
  • [8] MIRZADEH S, FARAJTABAR M, LI A, Et al., Improved knowledge distillation via teacher assistant, Proc of Association for the Advance of Artificial Intelligence (AAAI), pp. 1846-1857, (2020)
  • [9] POLINO A, PASCANU R, ALISTARH D., Model compression via distillation and quantization, Proc of International Conference on Learning Representations (ICLR), pp. 3293-3305, (2018)
  • [10] GU J, TRESP V., Search for better students to learn distilled knowledge, Proc of 24th European Conference on Artificial Intelligence (ECAI), pp. 6235-6246, (2020)
12