Empirical Estimation of Information Measures: A Literature Guide

被引：33
|
作者：
Verdu, Sergio
机构：
[1] Princeton, 08540, NJ
来源：
ENTROPY | 2019年 / 21卷 / 08期
基金：
美国国家科学基金会;
关键词：
information measures; empirical estimators; entropy; relative entropy; mutual information; universal estimation; DIVERGENCE ESTIMATION; ENTROPY ESTIMATION; NONPARAMETRIC-ESTIMATION; CONVERGENCE PROPERTIES; MUTUAL INFORMATION; RELATIVE ENTROPY; FUNCTIONALS; DISTRIBUTIONS; CLASSIFICATION; DENSITIES;
D O I：
10.3390/e21080720
中图分类号：
O4 [物理学];
学科分类号：
0702 ;
摘要：
We give a brief survey of the literature on the empirical estimation of entropy, differential entropy, relative entropy, mutual information and related information measures. While those quantities are of central importance in information theory, universal algorithms for their estimation are increasingly important in data science, machine learning, biology, neuroscience, economics, language, and other experimental sciences.
引用
收藏
页数：16
相关论文
共 50 条
  • [1] Information measures of kernel estimation
    Beheshti, Neshat
    Racine, Jeffrey S.
    Soofi, Ehsan S.
    ECONOMETRIC REVIEWS, 2019, 38 (01) : 47 - 68
  • [2] Minimax Estimation of Information Measures
    Jiao, Jiantao
    Venkat, Kartik
    Han, Yanjun
    Weissman, Tsachy
    2015 IEEE INTERNATIONAL SYMPOSIUM ON INFORMATION THEORY (ISIT), 2015, : 2296 - 2300
  • [3] Universal estimation of information measures
    Verdú, S
    PROCEEDINGS OF THE IEEE ITSOC INFORMATION THEORY WORKSHOP 2005 ON CODING AND COMPLEXITY, 2005, : 232 - 232
  • [4] EMPIRICAL STUDIES OF GASOLINE RETAILING: A GUIDE TO THE LITERATURE
    Eckert, Andrew
    JOURNAL OF ECONOMIC SURVEYS, 2013, 27 (01) : 140 - 166
  • [5] Risk measures, distortion parameters, and their empirical estimation
    Jones, Bruce L.
    Zitikis, Ricardas
    INSURANCE MATHEMATICS & ECONOMICS, 2007, 41 (02): : 279 - 297
  • [6] The Interplay Between Information and Estimation Measures
    Guo, Dongning
    Shamai , Shlomo
    Verdu, Sergio
    FOUNDATIONS AND TRENDS IN SIGNAL PROCESSING, 2012, 6 (04): : I - +
  • [7] CRITERION OF ESTIMATION BASED ON MEASURES OF INFORMATION
    PAPAIOAN.T
    ANNALS OF MATHEMATICAL STATISTICS, 1971, 42 (06): : 2179 - &
  • [8] Maximum Likelihood Estimation of Information Measures
    Jiao, Jiantao
    Venkat, Kartik
    Han, Yanjun
    Weissman, Tsachy
    2015 IEEE INTERNATIONAL SYMPOSIUM ON INFORMATION THEORY (ISIT), 2015, : 839 - 843
  • [9] Convergence of Smoothed Empirical Measures With Applications to Entropy Estimation
    Goldfeld, Ziv
    Greenewald, Kristjan
    Niles-Weed, Jonathan
    Polyanskiy, Yury
    IEEE TRANSACTIONS ON INFORMATION THEORY, 2020, 66 (07) : 4368 - 4391
  • [10] Information processing and PTSD: A review of the empirical literature
    Buckley, TC
    Blanchard, EB
    Neill, WT
    CLINICAL PSYCHOLOGY REVIEW, 2000, 20 (08) : 1041 - 1065
12345