Intrinsic Dimensionality Estimation of High-Dimension, Low Sample Size Data with D-Asymptotics

被引：15
|
作者：
Yata, Kazuyoshi [2 ]
Aoshima, Makoto [1 ]
机构：
[1] Univ Tsukuba, Inst Math, Ibaraki 3058571, Japan
[2] Univ Tsukuba, Grad Sch Pure & Appl Sci, Ibaraki 3058571, Japan
来源：
COMMUNICATIONS IN STATISTICS-THEORY AND METHODS | 2010年 / 39卷 / 8-9期
基金：
日本学术振兴会;
关键词：
Dual covariance matrix; Effective dimension; HDLSS; Large p small n; Maximum eigenvalue; GEOMETRIC REPRESENTATION; LARGEST EIGENVALUE;
D O I：
10.1080/03610920903121999
中图分类号：
O21 [概率论与数理统计]; C8 [统计学];
学科分类号：
020208 ; 070103 ; 0714 ;
摘要：
High-dimension, low sample size (HDLSS) data are becoming common in various fields such as genetic microarrays, medical imaging, text recognition, finance, chemometrics, and so on. Such data have surprising and often counterintuitive geometric structures because of the high-dimensional noise that dominates and corrupts the local neighborhoods. In this article, we estimate the intrinsic dimension (ID) that allows one to distinguish between deterministic chaos and random noise of HDLSS data. A new ID estimating methodology is given and its properties are studied by using a d-asymptotic approach.
引用
收藏
页码：1511 / 1521
页数：11
相关论文
共 50 条
  • [1] Classification for high-dimension low-sample size data
    Shen, Liran
    Er, Meng Joo
    Yin, Qingbo
    [J]. PATTERN RECOGNITION, 2022, 130
  • [2] Classification for high-dimension low-sample size data
    Shen, Liran
    Er, Meng Joo
    Yin, Qingbo
    [J]. PATTERN RECOGNITION, 2022, 130
  • [3] Statistical Significance of Clustering for High-Dimension, Low-Sample Size Data
    Liu, Yufeng
    Hayes, David Neil
    Nobel, Andrew
    Marron, J. S.
    [J]. JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION, 2008, 103 (483) : 1281 - 1293
  • [4] A survey of high dimension low sample size asymptotics
    Aoshima, Makoto
    Shen, Dan
    Shen, Haipeng
    Yata, Kazuyoshi
    Zhou, Yi-Hui
    Marron, J. S.
    [J]. AUSTRALIAN & NEW ZEALAND JOURNAL OF STATISTICS, 2018, 60 (01) : 4 - 19
  • [5] Robust Dimensionality Reduction for High-Dimension Data
    Xu, Huan
    Caramanis, Constantine
    Mannor, Shie
    [J]. 2008 46TH ANNUAL ALLERTON CONFERENCE ON COMMUNICATION, CONTROL, AND COMPUTING, VOLS 1-3, 2008, : 1291 - +
  • [6] Unsupervised classification of high-dimension and low-sample data with variational autoencoder based dimensionality reduction
    Mahmud, Mohammad Sultan
    Fu, Xianghua
    [J]. 2019 IEEE 4TH INTERNATIONAL CONFERENCE ON ADVANCED ROBOTICS AND MECHATRONICS (ICARM 2019), 2019, : 498 - 503
  • [7] Random forest kernel for high-dimension low sample size classification
    Cavalheiro, Lucca Portes
    Bernard, Simon
    Barddal, Jean Paul
    Heutte, Laurent
    [J]. STATISTICS AND COMPUTING, 2024, 34 (01)
  • [8] Random forest kernel for high-dimension low sample size classification
    Lucca Portes Cavalheiro
    Simon Bernard
    Jean Paul Barddal
    Laurent Heutte
    [J]. Statistics and Computing, 2024, 34
  • [9] High dimension low sample size asymptotics of robust PCA
    Zhou, Yi-Hui
    Marron, J. S.
    [J]. ELECTRONIC JOURNAL OF STATISTICS, 2015, 9 (01): : 204 - 218
  • [10] THE STATISTICS AND MATHEMATICS OF HIGH DIMENSION LOW SAMPLE SIZE ASYMPTOTICS
    Shen, Dan
    Shen, Haipeng
    Zhu, Hongtu
    Marron, J. S.
    [J]. STATISTICA SINICA, 2016, 26 (04) : 1747 - 1770
12345