Mutual information of several random variables and its estimation via variation

被引：3
|
作者：
Prelov, V. V. [1 ]
机构：
[1] RAS, Kharkevich Inst Informat Transmiss Problems, Moscow, Russia
来源：
PROBLEMS OF INFORMATION TRANSMISSION | 2009年 / 45卷 / 04期
基金：
俄罗斯基础研究基金会;
关键词：
INEQUALITY;
D O I：
10.1134/S0032946009040012
中图分类号：
TP301 [理论、方法];
学科分类号：
081202 ;
摘要：
We obtain some upper and lower bounds for the maximum of mutual information of several random variables via variational distance between the joint distribution of these random variables and the product of its marginal distributions. In this connection, some properties of variational distance between probability distributions of this type are derived. We show that in some special cases estimates of the maximum of mutual information obtained here are optimal or asymptotically optimal. Some results of this paper generalize the corresponding results of [1-3] to the multivariate case.
引用
收藏
页码：295 / 308
页数：14
相关论文
共 50 条
  • [1] Mutual information of several random variables and its estimation via variation
    V. V. Prelov
    [J]. Problems of Information Transmission, 2009, 45 : 295 - 308
  • [2] On mutual information estimation for mixed-pair random variables
    Beknazaryan, Aleksandr
    Dang, Xin
    Sang, Hailin
    [J]. STATISTICS & PROBABILITY LETTERS, 2019, 148 : 9 - 16
  • [3] Evaluation of Statistical Relationship of Random Variables via Mutual Information
    V. V. Tsurko
    A. I. Mikhalskii
    [J]. Automation and Remote Control, 2022, 83 : 734 - 742
  • [4] Evaluation of Statistical Relationship of Random Variables via Mutual Information
    Tsurko, V. V.
    Mikhalskii, A., I
    [J]. AUTOMATION AND REMOTE CONTROL, 2022, 83 (05) : 734 - 742
  • [5] On the Mutual Information between Random Variables in Networks
    Xu, Xiaoli
    Thakor, Satyajit
    Guan, Yong Liang
    [J]. 2013 IEEE INFORMATION THEORY WORKSHOP (ITW), 2013,
  • [6] The Mutual Information in Random Linear Estimation
    Barbier, Jean
    Dia, Mohamad
    Macris, Nicolas
    Krzakala, Florent
    [J]. 2016 54TH ANNUAL ALLERTON CONFERENCE ON COMMUNICATION, CONTROL, AND COMPUTING (ALLERTON), 2016, : 625 - 632
  • [7] Mutual Information Estimation with Random Forests
    Koeman, Mike
    Heskes, Tom
    [J]. NEURAL INFORMATION PROCESSING (ICONIP 2014), PT II, 2014, 8835 : 524 - 531
  • [8] Efficient Estimation of Mutual Information for Strongly Dependent Variables
    Gao, Shuyang
    Ver Steeg, Greg
    Galstyan, Aram
    [J]. ARTIFICIAL INTELLIGENCE AND STATISTICS, VOL 38, 2015, 38 : 277 - 286
  • [9] On estimation of information via variation
    Pinsker M.S.
    [J]. Problems of Information Transmission, 2005, 41 (2) : 71 - 75
  • [10] A Bayesian Alternative to Mutual Information for the Hierarchical Clustering of Dependent Random Variables
    Marrelec, Guillaume
    Messe, Arnaud
    Bellec, Pierre
    [J]. PLOS ONE, 2015, 10 (09):
12345