Higher-order Boltzmann machines and entropy bounds

被引:0
|
作者
Apolloni, B [1 ]
Battistini, E
de Falco, D
机构
[1] Univ Milan, Dipartimento Sci Informaz, I-20122 Milan, Italy
[2] Politecn Milan, Dipartimento Matemat, Milan, Italy
来源
JOURNAL OF PHYSICS A-MATHEMATICAL AND GENERAL | 1999年 / 32卷 / 30期
关键词
D O I
10.1088/0305-4470/32/30/301
中图分类号
O4 [物理学];
学科分类号
0702 ;
摘要
We examine some aspects of the interface area between mathematical statistics and statistical physics relevant to the study of Boltzmann machines. The Boltzmann machine learning algorithm is based on a variational principle (Gibbs' lemma for relative entropy). This fact suggests the possibility of a scheme of successive approximations: here we consider successive approximations parametrized by the order of many-body interactions among individual units. We prove bounds on the gain in relative entropy in the crucial step of adding, and estimating by Hebb's rule, a new parameter. We address the problem of providing, on the basis of local observations, upper and lower bounds on the entropy. While upper bounds are easily obtained by subadditivity, lower bounds involve localization of Hirschman bounds on a dual quantum system.
引用
收藏
页码:5529 / 5538
页数:10
相关论文
共 50 条