Origin of the computational hardness for learning with binary synapses

被引:39
|
作者
Huang, Haiping [1 ]
Kabashima, Yoshiyuki [1 ]
机构
[1] Tokyo Inst Technol, Dept Computat Intelligence & Syst Sci, Yokohama, Kanagawa 2268502, Japan
来源
PHYSICAL REVIEW E | 2014年 / 90卷 / 05期
关键词
NEURAL-NETWORK; GLASSY SYSTEMS; STATES;
D O I
10.1103/PhysRevE.90.052813
中图分类号
O35 [流体力学]; O53 [等离子体物理学];
学科分类号
070204 ; 080103 ; 080704 ;
摘要
Through supervised learning in a binary perceptron one is able to classify an extensive number of random patterns by a proper assignment of binary synaptic weights. However, to find such assignments in practice is quite a nontrivial task. The relation between the weight space structure and the algorithmic hardness has not yet been fully understood. To this end, we analytically derive the Franz-Parisi potential for the binary perceptron problem by starting from an equilibrium solution of weights and exploring the weight space structure around it. Our result reveals the geometrical organization of the weight space; the weight space is composed of isolated solutions, rather than clusters of exponentially many close-by solutions. The pointlike clusters far apart from each other in the weight space explain the previously observed glassy behavior of stochastic local search heuristics.
引用
收藏
页数:7
相关论文
共 50 条
  • [1] Generalization Learning in a Perceptron with Binary Synapses
    Baldassi, Carlo
    [J]. JOURNAL OF STATISTICAL PHYSICS, 2009, 136 (05) : 902 - 916
  • [2] Stochastic Hebbian learning with binary synapses
    Barrows, GL
    [J]. IEEE WORLD CONGRESS ON COMPUTATIONAL INTELLIGENCE, 1998, : 525 - 530
  • [3] Generalization Learning in a Perceptron with Binary Synapses
    Carlo Baldassi
    [J]. Journal of Statistical Physics, 2009, 136 : 902 - 916
  • [4] On the Hardness of Learning With Errors with Binary Secrets
    Micciancio, Daniele
    [J]. THEORY OF COMPUTING, 2018, 14 : 1 - 17
  • [5] Efficient supervised learning in networks with binary synapses
    Carlo Baldassi
    Alfredo Braunstein
    Nicolas Brunel
    Riccardo Zecchina
    [J]. BMC Neuroscience, 8 (Suppl 2)
  • [6] Efficient supervised learning in networks with binary synapses
    Baldassi, Carlo
    Braunstein, Alfredo
    Brunel, Nicolas
    Zecchina, Riccardo
    [J]. PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA, 2007, 104 (26) : 11079 - 11084
  • [7] LEARNING ALGORITHM FOR A NEURAL NETWORK WITH BINARY SYNAPSES
    KOHLER, H
    DIEDERICH, S
    KINZEL, W
    OPPER, M
    [J]. ZEITSCHRIFT FUR PHYSIK B-CONDENSED MATTER, 1990, 78 (02): : 333 - 342
  • [8] Convergence of stochastic learning in perceptrons with binary synapses
    Senn, W
    Fusi, S
    [J]. PHYSICAL REVIEW E, 2005, 71 (06):
  • [9] Morphological Learning in Multicompartment Neuron Model with Binary Synapses
    Hussain, Shaista
    Basu, Arindam
    [J]. 2016 IEEE INTERNATIONAL SYMPOSIUM ON CIRCUITS AND SYSTEMS (ISCAS), 2016, : 2527 - 2530
  • [10] Optimizing one-shot learning with binary Synapses
    Romani, Sandro
    Amit, Daniel J.
    Amit, Yali
    [J]. NEURAL COMPUTATION, 2008, 20 (08) : 1928 - 1950