Empirical Estimation of Information Measures: A Literature Guide

被引：33
|
作者：
Verdu, Sergio
机构：
[1] Princeton, 08540, NJ
来源：
ENTROPY | 2019年 / 21卷 / 08期
基金：
美国国家科学基金会;
关键词：
information measures; empirical estimators; entropy; relative entropy; mutual information; universal estimation; DIVERGENCE ESTIMATION; ENTROPY ESTIMATION; NONPARAMETRIC-ESTIMATION; CONVERGENCE PROPERTIES; MUTUAL INFORMATION; RELATIVE ENTROPY; FUNCTIONALS; DISTRIBUTIONS; CLASSIFICATION; DENSITIES;
D O I：
10.3390/e21080720
中图分类号：
O4 [物理学];
学科分类号：
0702 ;
摘要：
We give a brief survey of the literature on the empirical estimation of entropy, differential entropy, relative entropy, mutual information and related information measures. While those quantities are of central importance in information theory, universal algorithms for their estimation are increasingly important in data science, machine learning, biology, neuroscience, economics, language, and other experimental sciences.
引用
收藏
页数：16
相关论文
共 50 条
  • [41] Ground surface paleotemperature reconstruction using information measures and empirical Bayes
    Woodbury, AD
    Ferguson, G
    GEOPHYSICAL RESEARCH LETTERS, 2006, 33 (06)
  • [42] AN EMPIRICAL STUDY OF THE MAXIMAL AND TOTAL INFORMATION COEFFICIENTS AND LEADING MEASURES OF DEPENDENCE
    Reshef, David N.
    Reshef, Yakir A.
    Sabeti, Pardis C.
    Mitzenmacher, Michael
    ANNALS OF APPLIED STATISTICS, 2018, 12 (01): : 123 - 155
  • [43] INFORMATION-SYSTEMS FAILURES - A SURVEY AND CLASSIFICATION OF THE EMPIRICAL LITERATURE
    LYYTINEN, K
    HIRSCHHEIM, R
    OXFORD SURVEYS IN INFORMATION TECHNOLOGY, 1987, 4 : 257 - 309
  • [44] Firms' information environment measures: a literature review with focus on causality
    von Koch, Christopher
    Willesson, Magnus
    MANAGERIAL FINANCE, 2020, 46 (11) : 1343 - 1372
  • [45] Estimation of the labor market information-an empirical study in Taiwan
    Yang, Ping-Yu
    Chou, Li-Chen
    Wang, Zhan-Ao
    JOURNAL OF INTELLIGENT & FUZZY SYSTEMS, 2019, 37 (03) : 3477 - 3487
  • [46] Localization estimation and global vs. local information measures
    Pennini, F.
    Plastino, A.
    PHYSICS LETTERS A, 2007, 365 (04) : 263 - 267
  • [47] Maximum likelihood estimation using the empirical Fisher information matrix
    Scott, WA
    JOURNAL OF STATISTICAL COMPUTATION AND SIMULATION, 2002, 72 (08) : 599 - 611
  • [48] Nonparametric Estimation of Information-Based Measures of Statistical Dispersion
    Kostal, Lubomir
    Pokora, Ondrej
    ENTROPY, 2012, 14 (07): : 1221 - 1233
  • [49] Doubly bounded exponential model: Some information measures and estimation
    Singh, Brijesh P.
    Das, Utpal Dhar
    Karakaya, Kadir
    Bakouch, Hassan S.
    Abba, Badamasi
    COMMUNICATIONS IN STATISTICS-THEORY AND METHODS, 2024, 53 (22) : 7842 - 7859
  • [50] Managerial Coaching: A Review of the Empirical Literature and Development of a Model to Guide Future Practice
    Beattie, Rona S.
    Kim, Sewon
    Hagen, Marcia S.
    Egan, Toby M.
    Ellinger, Andrea D.
    Hamlin, Robert G.
    ADVANCES IN DEVELOPING HUMAN RESOURCES, 2014, 16 (02) : 184 - 201
12345