Efficiency of nearest neighbor entropy estimators for Bernoulli measures

被引：0
|
作者：
Timofeev, Evgeniy A. [2 ]
Kaltchenko, Alexei [1 ]
机构：
[1] Wilfrid Laurier Univ, Dept Phys & Comp, Waterloo, ON N2L 3C5, Canada
[2] Yaroslavl State Univ, Dept Theoret Informat, Yaroslavl, Russia
来源：
INDEPENDENT COMPONENT ANALYSES, COMPRESSIVE SAMPLING, WAVELETS, NEURAL NET, BIOSYSTEMS, AND NANOENGINEERING XII | 2014年 / 9118卷
关键词：
Metric; entropy; estimation; nonparametric; nearest neighbor; bias;
D O I：
10.1117/12.2049574
中图分类号：
TM [电工技术]; TN [电子技术、通信技术];
学科分类号：
0808 ; 0809 ;
摘要：
A problem of nonparametric entropy estimation for discrete stationary ergodic processes is considered. The estimation is based on so-called "nearest-neighbor method". It is shown that, for Bernoulli measures, the estimator is unbiased, i.e. converges to the (inverse) entropy of the process. Moreover, for symmetric Bernoulli measures, the unbiased estimator can be explicitly constructed.
引用
收藏
页数：5
相关论文
共 50 条
  • [11] Rate of convergence of the nearest neighbor entropy estimator
    Kaltchenko, A.
    Timofeeva, N.
    AEU-INTERNATIONAL JOURNAL OF ELECTRONICS AND COMMUNICATIONS, 2010, 64 (01) : 75 - 79
  • [12] Bias Reduction of the Nearest Neighbor Entropy Estimator
    Kaltchenko, Alexei
    Yang, En-hui
    Timofeeva, Nina
    2008 IEEE REGION 8 INTERNATIONAL CONFERENCE ON COMPUTATIONAL TECHNOLOGIES IN ELECTRICAL AND ELECTRONICS ENGINEERING: SIBIRCON 2008, PROCEEDINGS, 2008, : 261 - +
  • [13] Model-calibrated k-nearest neighbor estimators
    Magnussen, Steen
    Tomppo, Erkki
    SCANDINAVIAN JOURNAL OF FOREST RESEARCH, 2016, 31 (02) : 183 - 193
  • [14] Demystifying Fixed k-Nearest Neighbor Information Estimators
    Gao, Weihao
    Oh, Sewoong
    Viswanath, Pramod
    IEEE TRANSACTIONS ON INFORMATION THEORY, 2018, 64 (08) : 5629 - 5661
  • [15] Demystifying Fixed k-Nearest Neighbor Information Estimators
    Gao, Weihao
    Oh, Sewoong
    Viswanath, Pramod
    2017 IEEE INTERNATIONAL SYMPOSIUM ON INFORMATION THEORY (ISIT), 2017, : 1267 - 1271
  • [16] CONSISTENCY PROPERTIES OF NEAREST NEIGHBOR DENSITY-FUNCTION ESTIMATORS
    MOORE, DS
    YACKEL, JW
    ANNALS OF STATISTICS, 1977, 5 (01): : 143 - 154
  • [17] The Law of the Iterated Logarithm for the Multivariate Nearest Neighbor Density Estimators
    Ralescu, S. S.
    Journal of Radioanalytical and Nuclear Chemistry, 1994, 183 (02)
  • [18] Nonparametric k-nearest-neighbor entropy estimator
    Lombardi, Damiano
    Pant, Sanjay
    PHYSICAL REVIEW E, 2016, 93 (01)
  • [19] Nearest Neighbor Estimates of Entropy for Multivariate Circular Distributions
    Misra, Neeraj
    Singh, Harshinder
    Hnizdo, Vladimir
    ENTROPY, 2010, 12 (05): : 1125 - 1144
  • [20] Entropy based Nearest Neighbor Search in High Dimensions
    Panigrahy, Rina
    PROCEEDINGS OF THE SEVENTHEENTH ANNUAL ACM-SIAM SYMPOSIUM ON DISCRETE ALGORITHMS, 2006, : 1186 - 1195
12345