A Discretization Algorithm for Meteorological Data and its Parallelization Based on Hadoop

被引：2
|
作者：
Liu, Chao [1 ]
Jin, Wen [1 ]
Yu, Yuting [1 ]
Qiu, Taorong [1 ]
Bai, Xiaoming [1 ]
Zou, Shuilong [1 ]
机构：
[1] Nanchang Inst Sci & Technol, Sch Elect & Informat Engn, Nanchang, Jiangxi, Peoples R China
来源：
2017 INTERNATIONAL CONFERENCE ON CLOUD TECHNOLOGY AND COMMUNICATION ENGINEERING (CTCE2017) | 2017年 / 910卷
关键词：
D O I：
10.1088/1742-6596/910/1/012011
中图分类号：
TM [电工技术]; TN [电子技术、通信技术];
学科分类号：
0808 ; 0809 ;
摘要：
In view of the large amount of meteorological observation data, the property is more and the attribute values are continuous values, the correlation between the elements is the need for the application of meteorological data, this paper is devoted to solving the problem of how to better discretize large meteorological data to more effectively dig out the hidden knowledge in meteorological data and research on the improvement of discretization algorithm for large scale data, in order to achieve data in the large meteorological data discretization for the follow-up to better provide knowledge to provide protection, a discretization algorithm based on information entropy and inconsistency of meteorological attributes is proposed and the algorithm is parallelized under Hadoop platform. Finally, the comparison test validates the effectiveness of the proposed algorithm for discretization in the area of meteorological large data.
引用
收藏
页数：7
相关论文
共 50 条
  • [41] An Enhanced Apriori Algorithm Using Hybrid Data Layout Based on Hadoop for Big Data Processing
    Rochd, Yassir
    Hafidi, Imad
    INTERNATIONAL JOURNAL OF COMPUTER SCIENCE AND NETWORK SECURITY, 2018, 18 (06): : 161 - 167
  • [42] An improved task scheduling algorithm based on cache locality and data locality in Hadoop
    Zhang, Peng
    Li, Chunlin
    Zhao, Yahui
    2016 17TH INTERNATIONAL CONFERENCE ON PARALLEL AND DISTRIBUTED COMPUTING, APPLICATIONS AND TECHNOLOGIES (PDCAT), 2016, : 244 - 249
  • [43] A Personalized Recommendation Algorithm Based on Hadoop
    Huang, Hao
    Huang, Jianqing
    Ziavras, Sotirios G.
    Lu, Yaojie
    PROCEEDINGS OF 2015 IEEE 5TH INTERNATIONAL CONFERENCE ON ELECTRONICS INFORMATION AND EMERGENCY COMMUNICATION, 2015, : 406 - 409
  • [44] Study on GSP Algorithm Based on Hadoop
    Li, Huanhuan
    Zhou, Xiaofeng
    Pan, Chaojun
    PROCEEDINGS OF 2015 IEEE 5TH INTERNATIONAL CONFERENCE ON ELECTRONICS INFORMATION AND EMERGENCY COMMUNICATION, 2015, : 321 - 324
  • [45] A Hybrid Recommendation Algorithm Based on Hadoop
    Lin, Kunhui
    Wang, Jingjin
    Wang, Meihong
    2014 PROCEEDINGS OF THE 9TH INTERNATIONAL CONFERENCE ON COMPUTER SCIENCE & EDUCATION (ICCSE 2014), 2014, : 540 - 543
  • [46] A new Algorithm for Data Discretization and Feature Selection
    Ribeiro, Marcela Xavier
    Traina, Agma J. M.
    Traina, Caetano, Jr.
    APPLIED COMPUTING 2008, VOLS 1-3, 2008, : 953 - 954
  • [47] GPU parallelization of the sequential matrix diagonalization algorithm and its application to high-dimensional data
    Manuel Carcenac
    Soydan Redif
    Server Kasap
    The Journal of Supercomputing, 2017, 73 : 3603 - 3634
  • [48] Discretization Algorithm for Incomplete Economic Information in Rough Set Based on Big Data
    Li, Xiangyang
    Shen, Yangyang
    SYMMETRY-BASEL, 2020, 12 (08):
  • [49] A Gaussian mixture model based discretization algorithm for associative classification of medical data
    Khanmohammadi, Sina
    Chou, Chun-An
    EXPERT SYSTEMS WITH APPLICATIONS, 2016, 58 : 119 - 129
  • [50] GPU parallelization of the sequential matrix diagonalization algorithm and its application to high-dimensional data
    Carcenac, Manuel
    Redif, Soydan
    Kasap, Server
    JOURNAL OF SUPERCOMPUTING, 2017, 73 (08): : 3603 - 3634
12345