Deep neural networks compression based on improved clustering

被引：2
|
作者：
Liu H. [1 ]
Wang Y. [1 ]
Ma Y. [1 ]
机构：
[1] School of Automation and Information Engineering, Xi'an University of Technology, Xi'an, 710048, Shaanxi
来源：
Kongzhi Lilun Yu Yingyong/Control Theory and Applications | 2019年 / 36卷 / 07期
关键词：
Deep network compression; Deep neural networks; K-Means++; Pruning;
D O I：
10.7641/CTA.2018.70592
中图分类号：
学科分类号：
摘要：
Deep neural networks are typically over-parametrized and there is significant redundancy for deep learning models, which results in a waste of both computation and memory usage. In order to solve the problem, a new method based on improved clustering to compress the deep neural network is proposed. First of all, the network is pruned after the normal training. Then through the K-Means++ clustering the clustering center of each layer is gotten to achieve weight sharing. After the first two steps network weight quantization are also performed. The experiments on LeNet, AlexNet and VGG-16 are carried out, in which the deep neural network are compressed by 30 to 40 times without any loss of precision. The experimental results show that the deep neural network achieves effective compression without loss of accuracy through the method based on improved clustering, which makes the deployment of deep network on the mobile end possible. © 2019, Editorial Department of Control Theory & Applications South China University of Technology. All right reserved.
引用
收藏
页码：1130 / 1136
页数：6
相关论文
共 25 条
  • [1] Zhao D., Shao K., Zhu Y., Et al., Review of deep reinforcement learning and discussions on the development of computer go, Control Theory & Applications, 33, 6, pp. 701-717, (2016)
  • [2] Liu Z., Wu X., Interpretation of the artificial intelligence technology behind Alphago, Control Theory & Applications, 33, 12, pp. 1685-1687, (2016)
  • [3] Denil M., Shakibi B., Dinh L., Et al., Predicting parameters in deep learning, International Conference on Neural Information Processing Systems, pp. 2148-2156, (2013)
  • [4] Pratt L.Y., Comparing biases for minimal network construction with back-propagation, International Conference on Neural Information Processing Systems, pp. 177-185, (1988)
  • [5] Cun Y.L., Denker J.S., Solla S.A., Optimal brain damage, International Conference on Neural Information Processing Systems, pp. 598-605, (1989)
  • [6] Hassibi B., Stork D.G., Second order derivatives for network pruning: Optimal brain surgeon, Advances in Neural Information Processing Systems, 5, pp. 164-171, (1993)
  • [7] Han S., Pool J., Tran J., Et al., Learning both weights and connections for efficient neural networks, International Conference on Neural Information Processing Systems, pp. 1135-1143, (2015)
  • [8] Nguyem H.V., Zhou K., Vemulapalli R., Cross-domain synthesis of medical images using efficient location-sensitive deep network, Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015, (2015)
  • [9] Hwang K., Sung W., Fixed-point feedforward deep neural network design using weights +1, 0, and-1, Signal Processing Systems, pp. 1-6, (2014)
  • [10] Anwar S., Hwang K., Sung W., Fixed point optimization of deep convolutional neural networks for object recognition, IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 1131-1135, (2015)
123