Uniform Partitioning of Data Grid for Association Detection

被引:4
|
作者
Mousavi, Ali [1 ]
Baraniuk, Richard G. [2 ]
机构
[1] Google Res, Kirkland, WA 98033 USA
[2] Rice Univ, Dept Elect & Comp Engn, Houston, TX 77005 USA
基金
美国国家科学基金会;
关键词
Microwave integrated circuits; Mutual information; Partitioning algorithms; Heuristic algorithms; Dynamic programming; Extraterrestrial measurements; Correlation; Detection; estimation; correlation; mutual information; k-nearest neighbor; MUTUAL INFORMATION; DEPENDENCE; ENTROPY;
D O I
10.1109/TPAMI.2020.3029487
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Inferring appropriate information from large datasets has become important. In particular, identifying relationships among variables in these datasets has far-reaching impacts. In this article, we introduce the uniform information coefficient (UIC), which measures the amount of dependence between two multidimensional variables and is able to detect both linear and non-linear associations. Our proposed UIC is inspired by the maximal information coefficient (MIC) [1].; however, the MIC was originally designed to measure dependence between two one-dimensional variables. Unlike the MIC calculation that depends on the type of association between two variables, we show that the UIC calculation is less computationally expensive and more robust to the type of association between two variables. The UIC achieves this by replacing the dynamic programming step in the MIC calculation with a simpler technique based on the uniform partitioning of the data grid. This computational efficiency comes at the cost of not maximizing the information coefficient as done by the MIC algorithm. We present theoretical guarantees for the performance of the UIC and a variety of experiments to demonstrate its quality in detecting associations.
引用
收藏
页码:1098 / 1107
页数:10
相关论文
共 50 条
  • [1] Sub-Grid Partitioning Algorithm for Distributed Outlier Detection on Big Data
    Sakr, Mohamed
    Atwa, Walid
    Keshk, Arabi
    [J]. PROCEEDINGS OF 2018 13TH INTERNATIONAL CONFERENCE ON COMPUTER ENGINEERING AND SYSTEMS (ICCES), 2018, : 252 - 257
  • [2] Geometrical interpretation for data partitioning on a grid architecture
    Bernardi, Dominique
    Cerin, Christophe
    Fkaier, Hazem
    Jemni, Mohamed
    Koskas, Michel
    [J]. HPDC-15: PROCEEDINGS OF THE 15TH IEEE INTERNATIONAL SYMPOSIUM ON HIGH PERFORMANCE DISTRIBUTED COMPUTING, 2005, : 355 - 356
  • [3] Ordinal outlier detection based on recursive uniform partitioning
    Chen, Gang
    Shi, Juan
    Cai, Yuan Li
    [J]. TRANSACTIONS OF THE INSTITUTE OF MEASUREMENT AND CONTROL, 2013, 35 (07) : 940 - 948
  • [4] Approximate approximations with data on a perturbed uniform grid
    Lanzara, F.
    Maz'ya, V.
    Schmidt, G.
    [J]. ZEITSCHRIFT FUR ANALYSIS UND IHRE ANWENDUNGEN, 2008, 27 (03): : 323 - 338
  • [5] A novel partitioning algorithm for power grid using community detection
    Wang, Tao
    Zou, Pan
    Zhou, Heng
    [J]. 2012 2ND INTERNATIONAL CONFERENCE ON APPLIED ROBOTICS FOR THE POWER INDUSTRY (CARPI), 2012, : 511 - 513
  • [6] PMU Placement with Power Grid Partitioning for Line Outage Detection
    Alhalaseh, Rana
    Tokel, Halil Alper
    Chakraborty, Subhodeep
    Alirezaei, Gholamreza
    Mathar, Rudolf
    [J]. 2019 4TH INTERNATIONAL CONFERENCE ON SMART AND SUSTAINABLE TECHNOLOGIES (SPLITECH), 2019, : 353 - 358
  • [7] Strategies for partitioning data in association rule mining
    Ahmed, S
    Coenen, R
    Leng, P
    [J]. RESEARCH AND DEVELOPMENT IN INTELLIGENT SYSTEMS XX, 2004, : 127 - 139
  • [8] Fusion and partitioning of data for the detection of concealed weapons
    Varshney, PK
    Ramac, L
    Slamani, MA
    Alford, MG
    Ferris, D
    [J]. FUSION'98: PROCEEDINGS OF THE INTERNATIONAL CONFERENCE ON MULTISOURCE-MULTISENSOR INFORMATION FUSION, VOLS 1 AND 2, 1998, : 901 - 905
  • [9] A grouping approach based on non-uniform binary grid partitioning for crowd evacuation simulation
    Liu, Hong
    Li, Yan
    Li, Wenhao
    Lu, Dianjie
    Zhang, Guijuan
    [J]. CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE, 2019, 31 (23):
  • [10] Abnormal-Trajectory Detection Method Based on Variable Grid Partitioning
    Chen, Chuanming
    Xu, Dongsheng
    Yu, Qingying
    Gong, Shan
    Shi, Gege
    Liu, Haoming
    Chen, Wen
    [J]. ISPRS INTERNATIONAL JOURNAL OF GEO-INFORMATION, 2023, 12 (02)