Outlier detection in astronomical data

被引:6
|
作者
Zhang, YX [1 ]
Luo, A [1 ]
Zhao, YH [1 ]
机构
[1] Chinese Acad Sci, Natl Astron Observ, Beijing 100864, Peoples R China
关键词
outlier-data mining-data mining applications-algorithms-exceptions;
D O I
10.1117/12.550998
中图分类号
V [航空、航天];
学科分类号
08 ; 0825 ;
摘要
Astronomical data sets have experienced an unprecedented and continuing growth in the volume, quality, and complexity over the past few years, driven by the advances in telescope, detector, and computer technology. Like many other fields, astronomy has become a very data rich science. Information content measured in multiple Terabytes, and even larger, multi Petabyte data sets are on the horizon. To cope with this data flood, Virtual Observatory (VO) federates data archives and services representing a new information infrastructure for astronomy of the 21st century and provides the platform to science discovery. Data mining promises to both make the scientific utilization of these data sets more effective and more complete, and to open completely new avenues of astronomical research. Technological problems range from the issues of database design and federation, to data mining and advanced visualization, leading to a new toolkit for astronomical research. This is similar to challenges encountered in other data intensive fields today. Outlier detection is of great importance. as one of four knowledge discovery tasks. The identification of outliers can often lead to the discovery of truly unexpected knowledge in various fields. Especially in astronomy, the great interest of astronomers is to discover unusual, rare or unknown types of astronomical objects or phenomena. The outlier detection approaches in large datasets correctly meet the need of astronomers. In this paper we provide an overview of some techniques for automated identification of outliers in multivariate data. Outliers often provide useful information. Their identification is important not only for improving the analysis but also for indicating anomalies which may require further investigation. The technique may be used in the process of data preprocessing and also be used for preselecting special object candidates.
引用
收藏
页码:521 / 529
页数:9
相关论文
共 50 条
  • [31] A Survey of Outlier Detection Algorithms for Data Streams
    Tamboli, Jinita
    Shukla, Madhu
    PROCEEDINGS OF THE 10TH INDIACOM - 2016 3RD INTERNATIONAL CONFERENCE ON COMPUTING FOR SUSTAINABLE GLOBAL DEVELOPMENT, 2016, : 3535 - 3540
  • [32] Outlier Detection in Sparse Data with Factorization Machines
    Zhu, Mengxiao
    Aggarwal, Charu C.
    Ma, Shuai
    Zhang, Hui
    Huai, Jinpeng
    CIKM'17: PROCEEDINGS OF THE 2017 ACM CONFERENCE ON INFORMATION AND KNOWLEDGE MANAGEMENT, 2017, : 817 - 826
  • [33] A Novel Outlier Detection Method for Multivariate Data
    Almardeny, Yahya
    Boujnah, Noureddine
    Cleary, Frances
    IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, 2022, 34 (09) : 4052 - 4062
  • [34] Adaptive Threshold for Outlier Detection on Data Streams
    Clark, James P.
    Liu, Zhen
    Japkowicz, Nathalie
    2018 IEEE 5TH INTERNATIONAL CONFERENCE ON DATA SCIENCE AND ADVANCED ANALYTICS (DSAA), 2018, : 41 - 49
  • [35] A Nonparametric Outlier Detection Method for Financial Data
    Qu Ji-lin
    Qin Wen
    Sai Ying
    Feng Yu-mei
    2009 INTERNATIONAL CONFERENCE ON MANAGEMENT SCIENCE & ENGINEERING (16TH), VOLS I AND II, CONFERENCE PROCEEDINGS, 2009, : 1442 - +
  • [36] OUTLIER DETECTION FOR MULTI-NETWORK DATA
    Dey, Pritam
    Zhang, Zhengwu
    Dunson, David B.
    arXiv, 2022,
  • [37] Outlier and anomaly pattern detection on data streams
    Cheong Hee Park
    The Journal of Supercomputing, 2019, 75 : 6118 - 6128
  • [38] Outlier detection in multivariate analytical chemical data
    Egan, WJ
    Mogan, SL
    ANALYTICAL CHEMISTRY, 1998, 70 (11) : 2372 - 2379
  • [39] Outlier Detection by Regression Diagnostics in Large Data
    Nurunnabi, A. A. M.
    Nasser, Mohammed
    INTERNATIONAL CONFERENCE ON FUTURE COMPUTER AND COMMUNICATIONS, PROCEEDINGS, 2009, : 246 - +
  • [40] Automated outlier detection and estimation of missing data
    Rhyu, Jinwook
    Bozinovski, Dragana
    Dubs, Alexis B.
    Mohan, Naresh
    Bende, Elizabeth M. Cummings
    Maloney, Andrew J.
    Nieves, Miriam
    Sangerman, Jose
    Lu, Amos E.
    Hong, Moo Sun
    Artamonova, Anastasia
    Ou, Rui Wen
    Barone, Paul W.
    Leung, James C.
    Wolfrum, Jacqueline M.
    Sinskey, Anthony J.
    Springs, Stacy L.
    Braatz, Richard D.
    COMPUTERS & CHEMICAL ENGINEERING, 2024, 180