BOUNDS FOR KULLBACK-LEIBLER DIVERGENCE

被引：0
|
作者：
Popescu, Pantelimon G. [1 ]
Dragomir, Sever S. [2 ]
Slusanschi, Emil I. [1 ]
Stanasila, Octavian N. [1 ]
机构：
[1] Univ Politech Bucharest, Fac Automat Control & Comp, Comp Sci & Engn Dept, Splaiul Independentei 313, Bucharest 060042 6, Romania
[2] Victoria Univ, Coll Engn & Sci, POB 14428, Melbourne, MC 8001, Australia
来源：
ELECTRONIC JOURNAL OF DIFFERENTIAL EQUATIONS | 2016年
关键词：
Entropy; bounds; refinements; generalization; INFORMATION-THEORY;
D O I：
暂无
中图分类号：
O29 [应用数学];
学科分类号：
070104 ;
摘要：
Entropy, conditional entropy and mutual information for discrete-valued random variables play important roles in the information theory. The purpose of this paper is to present new bounds for relative entropy D(p parallel to q) of two probability distributions and then to apply them to simple entropy and mutual information. The relative entropy upper bound obtained is a refinement of a bound previously presented into literature.
引用
收藏
页数：6
相关论文
共 50 条
  • [1] Guaranteed Bounds on the Kullback-Leibler Divergence of Univariate Mixtures
    Nielsen, Frank
    Sun, Ke
    [J]. IEEE SIGNAL PROCESSING LETTERS, 2016, 23 (11) : 1543 - 1546
  • [2] Renyi Divergence and Kullback-Leibler Divergence
    van Erven, Tim
    Harremoes, Peter
    [J]. IEEE TRANSACTIONS ON INFORMATION THEORY, 2014, 60 (07) : 3797 - 3820
  • [3] The fractional Kullback-Leibler divergence
    Alexopoulos, A.
    [J]. JOURNAL OF PHYSICS A-MATHEMATICAL AND THEORETICAL, 2021, 54 (07)
  • [4] New bounds for the empirical robust Kullback-Leibler divergence problem
    Bahceci, Ufuk
    [J]. INFORMATION SCIENCES, 2023, 637
  • [5] Kullback-Leibler Divergence Revisited
    Raiber, Fiana
    Kurland, Oren
    [J]. ICTIR'17: PROCEEDINGS OF THE 2017 ACM SIGIR INTERNATIONAL CONFERENCE THEORY OF INFORMATION RETRIEVAL, 2017, : 117 - 124
  • [6] On the Interventional Kullback-Leibler Divergence
    Wildberger, Jonas
    Guo, Siyuan
    Bhattacharyya, Arnab
    Schoelkopf, Bernhard
    [J]. CONFERENCE ON CAUSAL LEARNING AND REASONING, VOL 213, 2023, 213 : 328 - 349
  • [7] Model Fusion with Kullback-Leibler Divergence
    Claici, Sebastian
    Yurochkin, Mikhail
    Ghosh, Soumya
    Solomon, Justin
    [J]. INTERNATIONAL CONFERENCE ON MACHINE LEARNING, VOL 119, 2020, 119
  • [8] Bounds on the Kullback-Leibler divergence rate between hidden Markov models
    Vidyasagar, M.
    [J]. PROCEEDINGS OF THE 46TH IEEE CONFERENCE ON DECISION AND CONTROL, VOLS 1-14, 2007, : 2196 - 2201
  • [9] Kullback-Leibler Divergence Metric Learning
    Ji, Shuyi
    Zhang, Zizhao
    Ying, Shihui
    Wang, Liejun
    Zhao, Xibin
    Gao, Yue
    [J]. IEEE TRANSACTIONS ON CYBERNETICS, 2022, 52 (04) : 2047 - 2058
  • [10] Nonparametric Estimation of Kullback-Leibler Divergence
    Zhang, Zhiyi
    Grabchak, Michael
    [J]. NEURAL COMPUTATION, 2014, 26 (11) : 2570 - 2593
12345