Big Data Cleaning

被引:0
|
作者
Tang, Nan [1 ]
机构
[1] Qatar Comp Res Inst, Doha, Qatar
关键词
D O I
暂无
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Data cleaning is, in fact, a lively subject that has played an important part in the history of data management and data analytics, and it still is undergoing rapid development. Moreover, data cleaning is considered as a main challenge in the era of big data, due to the increasing volume, velocity and variety of data in many applications. This paper aims to provide an overview of recent work in different aspects of data cleaning: error detection methods, data repairing algorithms, and a generalized data cleaning system. It also includes some discussion about our efforts of data cleaning methods from the perspective of big data, in terms of volume, velocity and variety.
引用
收藏
页码:13 / 24
页数:12
相关论文
共 50 条
  • [21] A SYSTEMATIC MAPPING REVIEW ON DATA CLEANING METHODS IN BIG DATA ENVIRONMENTS
    Iwata, Claudio Keiji
    Galegale, Napoleao Verardi
    Ito, Marcia
    de Azevedo, Marilia Macorin
    Feitosa, Marcelo Duduchi
    Arima, Carlos Hideo
    IADIS-INTERNATIONAL JOURNAL ON COMPUTER SCIENCE AND INFORMATION SYSTEMS, 2024, 19 (02): : 19 - 36
  • [22] An Incorrect Data Detection Method for Big Data Cleaning of Machinery Condition Monitoring
    Xu, Xuefang
    Lei, Yaguo
    Li, Zeda
    IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, 2020, 67 (03) : 2326 - 2336
  • [23] A data cleaning model for electric power big data based on Spark framework
    Qu, Zhao-Yang
    Wang, Yong-Wen
    Wang, Chong
    Qu, Nan
    Yan, Jia
    International Journal of Database Theory and Application, 2016, 9 (03): : 137 - 150
  • [24] Data Cleaning Optimization for Grain Big Data Processing using Task Merging
    Ju, Xingang
    Lian, Feiyu
    Zhang, Yuan
    2019 6TH INTERNATIONAL CONFERENCE ON INFORMATION SCIENCE AND CONTROL ENGINEERING (ICISCE 2019), 2019, : 225 - 233
  • [25] Experimental Optimization of Big Data Cleaning Method for Agricultural Machinery
    Yuan Y.
    Xu L.
    Ji F.
    Guo D.
    An S.
    Niu K.
    Nongye Jixie Xuebao/Transactions of the Chinese Society for Agricultural Machinery, 2021, 52 (06): : 35 - 42
  • [26] CLEANING AND ANALYSIS OF ROAD-SENSOR-RECORDED BIG DATA
    Almansour, Aseel
    ADVANCES AND APPLICATIONS IN STATISTICS, 2021, 69 (01) : 7 - 22
  • [27] Data Cleaning for Accurate, Fair, and Robust Models: A Big Data - AI Integration Approach
    Tae, Ki Hyun
    Roh, Yuji
    Oh, Young Hun
    Kim, Hyunsu
    Whang, Steven Euijong
    PROCEEDINGS OF THE 3RD INTERNATIONAL WORKSHOP ON DATA MANAGEMENT FOR END-TO-END MACHINE LEARNING, DEEM 2019, 2019,
  • [28] A Big Data Online Cleaning Algorithm Based on Dynamic Outlier Detection
    Diao, Yinglong
    Liu, Ke-yan
    Meng, Xiaoli
    Ye, Xueshun
    He, Kaiyuan
    2015 INTERNATIONAL CONFERENCE ON CYBER-ENABLED DISTRIBUTED COMPUTING AND KNOWLEDGE DISCOVERY, 2015, : 230 - 234
  • [29] Research on the Construction of Economic Statistical Model and Application of Data Cleaning Technology in Big Data Environment
    Peng, Ziying
    Applied Mathematics and Nonlinear Sciences, 2024, 9 (01)
  • [30] The cleaning method of duplicate big data based on association rule mining algorithm
    Wu, Ming
    INTERNATIONAL JOURNAL OF AUTONOMOUS AND ADAPTIVE COMMUNICATIONS SYSTEMS, 2023, 16 (02) : 220 - 231