Nonproduct Data-Dependent Partitions for Mutual Information Estimation: Strong Consistency and Applications

被引：20

作者：

Silva, Jorge ^{[1
]}

Narayanan, Shrikanth ^{[2
]}

机构：

[1] Univ Chile, Dept Elect Engn, Santiago 4123, Chile

[2] Univ So Calif, Viterbi Sch Engn, Dept Elect Engn, Los Angeles, CA 90089 USA

来源：

IEEE TRANSACTIONS ON SIGNAL PROCESSING | 2010年 / 58卷 / 07期

基金：

美国国家科学基金会;

关键词：

Asymptotically sufficient partitions; data-dependent partitions; histogram-based estimation; mutual information; tree-structured vector quantization; Vapnik-Chervonenkis inequality; CONVERGENCE PROPERTIES; PATTERN-RECOGNITION; DENSITY-ESTIMATION; CLASSIFICATION; OPTIMIZATION; PROBABILITY; ENTROPY; TREES; L1;

D O I：

10.1109/TSP.2010.2046077

中图分类号：

TM [电工技术]; TN [电子技术、通信技术];

学科分类号：

0808 ; 0809 ;

摘要：

A new framework for histogram-based mutual information estimation of probability distributions equipped with density functions in (R(d), B(R(d))) is presented in this work. A general histogram-based estimate is proposed, considering non-product data-dependent partitions, and sufficient conditions are stipulated to guarantee a strongly consistent estimate for mutual information. Two emblematic families of density-free strongly consistent estimates are derived from this result, one based on statistically equivalent blocks (the Gessaman's partition) and the other, on a tree-structured vector quantization scheme.

引用

页码：3497 / 3511

页数：15

共 50 条

[1] Information divergence estimation based on data-dependent partitions
Silva, Jorge
Narayanan, Shrikanth S.
[J]. JOURNAL OF STATISTICAL PLANNING AND INFERENCE, 2010, 140 (11) : 3180 - 3198
[2] Histogram regression estimation using data-dependent partitions
Nobel, A
[J]. ANNALS OF STATISTICS, 1996, 24 (03): : 1084 - 1105
[3] Divergence estimation of continuous distributions based on data-dependent partitions
Wang, Q
Kulkarni, SR
Verdú, S
[J]. IEEE TRANSACTIONS ON INFORMATION THEORY, 2005, 51 (09) : 3064 - 3074
[4] Universal estimation of divergence for continuous distributions via data-dependent partitions
Wang, Q
Kulkarni, SR
Verdú, S
[J]. 2005 IEEE INTERNATIONAL SYMPOSIUM ON INFORMATION THEORY (ISIT), VOLS 1 AND 2, 2005, : 152 - 156
[5] MULTIVARIATE HISTOGRAMS WITH DATA-DEPENDENT PARTITIONS
Klemela, Jussi
[J]. STATISTICA SINICA, 2009, 19 (01) : 159 - 176
[6] A HIGH DIMENSIONAL DIRECTED INFORMATION ESTIMATION USING DATA-DEPENDENT PARTITIONING
Liu, Ying
Aviyente, Selin
Al-khassaweneh, Mahmood
[J]. 2009 IEEE/SP 15TH WORKSHOP ON STATISTICAL SIGNAL PROCESSING, VOLS 1 AND 2, 2009, : 605 - +
[7] Mutual Information Estimation: Independence Detection and Consistency
Suzuki, Joe
[J]. 2019 IEEE INTERNATIONAL SYMPOSIUM ON INFORMATION THEORY (ISIT), 2019, : 2514 - 2518
[8] STRONG CONSISTENCY OF DENSITY ESTIMATION BY ORTHOGONAL SERIES METHODS FOR DEPENDENT VARIABLES WITH APPLICATIONS
AHMAD, IA
[J]. ANNALS OF THE INSTITUTE OF STATISTICAL MATHEMATICS, 1979, 31 (02) : 279 - 288
[9] Universal consistency of data-driven partitions for divergence estimation
Silva, Jorge
Narayanan, Shrikanth
[J]. 2007 IEEE INTERNATIONAL SYMPOSIUM ON INFORMATION THEORY PROCEEDINGS, VOLS 1-7, 2007, : 2021 - 2025
[10] Data-Dependent Label Distribution Learning for Age Estimation
He, Zhouzhou
Li, Xi
Zhang, Zhongfei
Wu, Fei
Geng, Xin
Zhang, Yaqing
Yang, Ming-Hsuan
Zhuang, Yueting
[J]. IEEE TRANSACTIONS ON IMAGE PROCESSING, 2017, 26 (08) : 3846 - 3858

← 1 2 3 4 5 →