Bayesian retrieval in associative memories with storage errors

被引:25
|
作者
Sommer, FT [1 ]
Dayan, P [1 ]
机构
[1] MIT, Dept Brain & Cognit Sci, Cambridge, MA 02139 USA
来源
基金
美国国家科学基金会;
关键词
Bayesian reasoning; correlation associative memory; graded response neurons; iterative retrieval; maximum likelihood retrieval; mean field methods; threshold strategies; storage errors; Willshaw model;
D O I
10.1109/72.701183
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
It is well known that for finite-sized networks, onestep retrieval in the autoassociative Willshaw net is a suboptimal way to extract the information stored in the synapses. Iterative retrieval strategies are much better, but have;hitherto only had heuristic justification. We show how they emerge naturally from considerations of probabilistic inference under conditions of noisy and partial input and a corrupted weight matrix. We start from the conditional probability distribution over possible patterns for retrieval, This contains all possible information that is available to an observer of the network and the initial input. Since this distribution is over exponentially many patterns, we use it to develop two approximate, but tractable, iterative retrieval methods. One performs maximum likelihood inference to find the single most likely pattern, using the (negative log of the) conditional probability as a Lyapunov function for retrieval, In physics terms, if storage errors are present, then the modified iterative update equations contain an additional antiferromagnetic interaction term and site dependent threshold values. The second method makes a mean field assumption to optimize a tractable estimate of the full conditional probability distribution. This leads to iterative mean field equations which can be interpreted in terms of a network of neurons with sigmoidal responses but with the same interactions and thresholds as in the maximum likelihood update equations, In the absence of storage errors, both models become very similiar to the WiIlshaw model, where standard retrieval is iterated using a particular form of linear threshold strategy.
引用
收藏
页码:705 / 713
页数:9
相关论文
共 50 条
  • [31] DISTRIBUTED AND FAULT-TOLERANT COMPUTATION FOR RETRIEVAL TASKS USING DISTRIBUTED ASSOCIATIVE MEMORIES
    CHAR, JM
    CHERKASSKY, V
    WECHSLER, H
    ZIMMERMAN, GL
    IEEE TRANSACTIONS ON COMPUTERS, 1988, 37 (04) : 484 - 490
  • [32] SUPERCONDUCTIVE ASSOCIATIVE MEMORIES
    AHRONS, RW
    RCA REVIEW, 1963, 24 (03): : 325 - 354
  • [33] Extended αβ associative memories
    Azuela, J. H. Sossa
    Fernandez, R. Barron
    REVISTA MEXICANA DE FISICA, 2007, 53 (01) : 10 - 20
  • [34] BIDIRECTIONAL ASSOCIATIVE MEMORIES
    KOSKO, B
    IEEE TRANSACTIONS ON SYSTEMS MAN AND CYBERNETICS, 1988, 18 (01): : 49 - 60
  • [35] COMBINED ASSOCIATIVE MEMORIES
    YONG, X
    OPTICS LETTERS, 1990, 15 (19) : 1091 - 1093
  • [36] Associative memories and processors
    Wolf, G.
    IT - Information Technology, 1975, 17 (1-6): : 264 - 271
  • [37] ASSOCIATIVE HOLOGRAPHIC MEMORIES
    GABOR, D
    IBM JOURNAL OF RESEARCH AND DEVELOPMENT, 1969, 13 (02) : 156 - &
  • [38] Episodic associative memories
    Hattori, M
    Hagiwara, M
    NEUROCOMPUTING, 1996, 12 (01) : 1 - 18
  • [39] Morphological associative memories
    Ritter, GX
    Sussner, P
    Diaz-de-Leon, JL
    IEEE TRANSACTIONS ON NEURAL NETWORKS, 1998, 9 (02): : 281 - 293
  • [40] Episodic associative memories
    Department of Electrical Engineering, Faculty of Science and Technology, Keio University, Yokohama-shi, 223, Japan
    NEUROCOMPUTING, 1 (1-18):