Compressing neural networks with two-layer decoupling

被引：0

作者：

De Jonghe, Joppe ^{[1
]}

Usevich, Konstantin ^{[2
]}

Dreesen, Philippe ^{[3
]}

Ishteva, Mariya ^{[1
]}

机构：

[1] Katholieke Univ Leuven, Dept Comp Sci, Geel, Belgium

[2] Univ Lorraine, CNRS, Nancy, France

[3] Maastricht Univ, DACS, Maastricht, Netherlands

来源：

2023 IEEE 9TH INTERNATIONAL WORKSHOP ON COMPUTATIONAL ADVANCES IN MULTI-SENSOR ADAPTIVE PROCESSING, CAMSAP | 2023年

关键词：

tensor; tensor decomposition; decoupling; compression; neural network; MODEL COMPRESSION; ACCELERATION;

D O I：

10.1109/CAMSAP58249.2023.10403509

中图分类号：

TP39 [计算机的应用];

学科分类号：

081203 ; 0835 ;

摘要：

The single-layer decoupling problem has recently been used for the compression of neural networks. However, methods that are based on the single-layer decoupling problem only allow the compression of a neural network to a single flexible layer. As a result, compressing more complex networks leads to worse approximations of the original network due to only having one flexible layer. Having the ability to compress to more than one flexible layer thus allows to better approximate the underlying network compared to compression into only a single flexible layer. Performing compression into more than one flexible layer corresponds to solving a multi-layer decoupling problem. As a first step towards general multi-layer decoupling, this work introduces a method for solving the two-layer decoupling problem in the approximate case. This method enables the compression of neural networks into two flexible layers.

引用

下载

页码：226 / 230

页数：5

共 50 条

[21] Synchronizability of two-layer networks
Xu, Mingming
Zhou, Jin
Lu, Jun-an
Wu, Xiaoqun
EUROPEAN PHYSICAL JOURNAL B, 2015, 88 (09):
[22] Two-layer queueing networks
Kino, I
JOURNAL OF THE OPERATIONS RESEARCH SOCIETY OF JAPAN, 1997, 40 (02) : 163 - 185
[23] Synchronizability of two-layer networks
Mingming Xu
Jin Zhou
Jun-an Lu
Xiaoqun Wu
The European Physical Journal B, 2015, 88
[24] On the Connection Between Learning Two-Layer Neural Networks and Tensor Decomposition
Mondelli, Marco
Montanari, Andrea
22ND INTERNATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE AND STATISTICS, VOL 89, 2019, 89
[25] An online gradient method with momentum for two-layer feedforward neural networks
Zhang, Naimin
APPLIED MATHEMATICS AND COMPUTATION, 2009, 212 (02) : 488 - 498
[26] Convergence of a Gradient Algorithm with Penalty for Training Two-layer Neural Networks
Shao, Hongmei
Liu, Lijun
Zheng, Gaofeng
2009 2ND IEEE INTERNATIONAL CONFERENCE ON COMPUTER SCIENCE AND INFORMATION TECHNOLOGY, VOL 4, 2009, : 76 - +
[27] Spurious Local Minima are Common in Two-Layer ReLU Neural Networks
Safran, Itay
Shamir, Ohad
INTERNATIONAL CONFERENCE ON MACHINE LEARNING, VOL 80, 2018, 80
[28] Convergence of gradient method with momentum for two-layer feedforward neural networks
Zhang, NM
Wu, W
Zheng, GF
IEEE TRANSACTIONS ON NEURAL NETWORKS, 2006, 17 (02): : 522 - 525
[29] The VC dimension and pseudodimension of two-layer neural networks with discrete inputs
Bartlett, PL
Williamson, RC
NEURAL COMPUTATION, 1996, 8 (03) : 625 - 628
[30] A New Propagator for Two-Layer Neural Networks in Empirical Model Learning
Lombardi, Michele
Gualandi, Stefano
PRINCIPLES AND PRACTICE OF CONSTRAINT PROGRAMMING, CP 2013, 2013, 8124 : 448 - 463

← 1 2 3 4 5 →