The limitations of differentiable architecture search

被引：0
|
作者：
Guillaume, Lacharme [1 ]
Hubert, Cardot [1 ]
Christophe, Lente [1 ]
Nicolas, Monmarche [1 ]
机构：
[1] Univ Tours, LIFAT, 64 Ave Jean Portalis, F-37200 Tours, France
来源：
PATTERN ANALYSIS AND APPLICATIONS | 2024年 / 27卷 / 02期
关键词：
Differentiable architecture search (DARTS); Bi-level optimization; Depth regularization;
D O I：
10.1007/s10044-024-01260-5
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
In this paper, we will provide a detailed explanation of the limitations behind differentiable architecture search (DARTS). Algorithms based on the DARTS paradigm tend to converge towards degenerate solutions. A degenerate solution corresponds to an architecture with a shallow graph containing mainly skip connections. We have identified 6 sources of errors that could explain this phenomenon. Some of these errors can only be partially eliminated. Therefore, we will propose an innovative solution to remove degenerate solutions from the search space. We will demonstrate the validity of our approach through experiments conducted on the CIFAR10 and CIFAR100 databases. Our code is available at the following link: https://scm.univ-tours.fr/projetspublics/lifat/darts_ibpria_sparcity
引用
下载
收藏
页数：14
相关论文
共 50 条
  • [21] Search Space Adaptation for Differentiable Neural Architecture Search in Image Classification
    Kim, Youngkee
    Jung, Soyi
    Choi, Minseok
    Kim, Joongheon
    2022 THIRTEENTH INTERNATIONAL CONFERENCE ON UBIQUITOUS AND FUTURE NETWORKS (ICUFN), 2022, : 363 - 365
  • [22] MergeNAS: Merge Operations into One for Differentiable Architecture Search
    Wang, Xiaoxing
    Xue, Chao
    Yan, Junchi
    Yang, Xiaokang
    Hu, Yonggang
    Sun, Kewei
    PROCEEDINGS OF THE TWENTY-NINTH INTERNATIONAL JOINT CONFERENCE ON ARTIFICIAL INTELLIGENCE, 2020, : 3065 - 3072
  • [23] Enhanced Differentiable Architecture Search Based on Asymptotic Regularization
    Jin, Cong
    Huang, Jinjie
    Chen, Yuanjian
    Gong, Yuqing
    CMC-COMPUTERS MATERIALS & CONTINUA, 2024, 78 (02): : 1547 - 1568
  • [24] NDARTS: A Differentiable Architecture Search Based on the Neumann Series
    Han, Xiaoyu
    Li, Chenyu
    Wang, Zifan
    Liu, Guohua
    ALGORITHMS, 2023, 16 (12)
  • [25] Mean-Shift Based Differentiable Architecture Search
    Hsieh J.-W.
    Chou C.-H.
    Chang M.-C.
    Chen P.-Y.
    Santra S.
    Huang C.-S.
    IEEE Transactions on Artificial Intelligence, 2024, 5 (03): : 1235 - 1246
  • [26] Differentiable neural architecture search with channel performance measurement
    Pan J.
    Zheng X.-C.
    Zou X.-Y.
    Kongzhi yu Juece/Control and Decision, 2024, 39 (07): : 2151 - 2160
  • [27] D-DARTS: Distributed Differentiable Architecture Search
    Heuillet, Alexandre
    Tabia, Hedi
    Arioui, Hichem
    Youcef-Toumi, Kamal
    PATTERN RECOGNITION LETTERS, 2023, 176 : 42 - 48
  • [28] DASS: Differentiable Architecture Search for Sparse Neural Networks
    Mousavi, Hamid
    Loni, Mohammad
    Alibeigi, Mina
    Daneshtalab, Masoud
    ACM TRANSACTIONS ON EMBEDDED COMPUTING SYSTEMS, 2023, 22 (05)
  • [29] D-DARTS: Distributed Differentiable Architecture Search
    Heuillet, Alexandre
    Tabia, Hedi
    Arioui, Hichem
    Youcef-Toumi, Kamal
    PATTERN RECOGNITION LETTERS, 2023, 176 : 42 - 48
  • [30] Operation and Topology Aware Fast Differentiable Architecture Search
    Siddiqui, Shahid
    Kyrkou, Christos
    Theocharides, Theocharis
    2020 25TH INTERNATIONAL CONFERENCE ON PATTERN RECOGNITION (ICPR), 2021, : 9666 - 9673
12345