The limitations of differentiable architecture search

被引：0

作者：

Guillaume, Lacharme ^{[1
]}

Hubert, Cardot ^{[1
]}

Christophe, Lente ^{[1
]}

Nicolas, Monmarche ^{[1
]}

机构：

[1] Univ Tours, LIFAT, 64 Ave Jean Portalis, F-37200 Tours, France

来源：

PATTERN ANALYSIS AND APPLICATIONS | 2024年 / 27卷 / 02期

关键词：

Differentiable architecture search (DARTS); Bi-level optimization; Depth regularization;

D O I：

10.1007/s10044-024-01260-5

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

In this paper, we will provide a detailed explanation of the limitations behind differentiable architecture search (DARTS). Algorithms based on the DARTS paradigm tend to converge towards degenerate solutions. A degenerate solution corresponds to an architecture with a shallow graph containing mainly skip connections. We have identified 6 sources of errors that could explain this phenomenon. Some of these errors can only be partially eliminated. Therefore, we will propose an innovative solution to remove degenerate solutions from the search space. We will demonstrate the validity of our approach through experiments conducted on the CIFAR10 and CIFAR100 databases. Our code is available at the following link: https://scm.univ-tours.fr/projetspublics/lifat/darts_ibpria_sparcity

引用

下载

页数：14

共 50 条

[41] Differentiable neural architecture search for domain adaptation in fault diagnosis
Liu, Yumeng
Li, Xudong
Hu, Yang
MECHANICAL SYSTEMS AND SIGNAL PROCESSING, 2023, 202
[42] Zero-Cost Operation Scoring in Differentiable Architecture Search
Xiang, Lichuan
Dudziak, Lukasz
Abdelfattah, Mohamed S.
Chau, Thomas
Lane, Nicholas D.
Wen, Hongkai
THIRTY-SEVENTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, VOL 37 NO 9, 2023, : 10453 - 10463
[43] Inner Loop-Based Modified Differentiable Architecture Search
Jin, Cong
Huang, Jinjie
IEEE ACCESS, 2024, 12 : 41918 - 41933
[44] AlphaGAN: Fully Differentiable Architecture Search for Generative Adversarial Networks
Tian, Yuesong
Shen, Li
Su, Guinan
Li, Zhifeng
Liu, Wei
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2022, 44 (10) : 6752 - 6766
[45] DMNAS: Differentiable Multi-modal Neural Architecture Search
Funoki, Yushiro
Ono, Satoshi
INTERNATIONAL WORKSHOP ON ADVANCED IMAGING TECHNOLOGY (IWAIT) 2021, 2021, 11766
[46] RAPDARTS: Resource-Aware Progressive Differentiable Architecture Search
Green, Sam
Vineyard, Craig M.
Helinski, Ryan
Koc, Cetin Kaya
2020 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), 2020,
[47] Inner Loop-Based Modified Differentiable Architecture Search
Jin, Cong
Huang, Jinjie
IEEE Access, 2024, 12 : 41918 - 41933
[48] DDNAS: Discretized Differentiable Neural Architecture Search for Text Classification
Chen, Kuan-Chun
Li, Cheng-Te
Lee, Kuo-Jung
ACM TRANSACTIONS ON INTELLIGENT SYSTEMS AND TECHNOLOGY, 2023, 14 (05)
[49] Improving Differentiable Architecture Search via self-distillation
Zhu, Xunyu
Li, Jian
Liu, Yong
Wang, Weiping
NEURAL NETWORKS, 2023, 167 : 656 - 667
[50] μDARTS: Model Uncertainty-Aware Differentiable Architecture Search
Chakraborty, Biswadeep
Mukhopadhyay, Saibal
IEEE ACCESS, 2022, 10 : 98670 - 98682

← 1 2 3 4 5 →