A MapReduce-based approach to social network big data mining

被引:1
|
作者
Qi, Fuli [1 ]
机构
[1] Shanghai Zhongqiao Vocat & Tech Univ, Sch Informat Engn, Shanghai 201514, Peoples R China
关键词
Social network; big data; MapReduce; parallel K-means clustering algorithm; Weibo topic; ALGORITHM;
D O I
10.3233/JCM-226903
中图分类号
T [工业技术];
学科分类号
08 ;
摘要
The rapid development of social networks has facilitated the convenience of users to receive information. As a network communication platform for people's daily use, microblog has countless information data. In view of the low efficiency and poor clustering effect of K-means algorithm, a parallel K-means clustering algorithm based on MapReduce model is studied; In order to alleviate the difficulty in calculating the similarity of microblog topic text, the space vector model and semantic similarity are used to calculate the similarity between texts to improve the quality of microblog text classification. The data expansion rate of corresponding nodes under different data sets shows that the average expansion rate of the parallel K-means algorithm reaches 0.89, and the running rate is the highest. The results show that the parallel K-means algorithm has good clustering stability and the highest clustering quality, reaching 1.24; The clustering time of the algorithm is the shortest, the average clustering time is 1.27 minutes, and the clustering effect and efficiency of the algorithm are the best. In the quality analysis of Weibo topic recommendation, the accuracy of P-K-means recommendation is 95.64%, user satisfaction is 98.64%, and the recommendation effect is also the best. It shows that the research on the parallel K-means clustering algorithm based on MapReduce has the best performance in microblogging topic mining and recommendation, which can efficiently recommend topics of interest to users and enhance users' microblogging experience.
引用
收藏
页码:2535 / 2547
页数:13
相关论文
共 50 条
  • [31] MapReduce-based big data classification model using feature subset selection and hyperparameter tuned deep belief network
    Surendran Rajendran
    Osamah Ibrahim Khalaf
    Youseef Alotaibi
    Saleh Alghamdi
    [J]. Scientific Reports, 11
  • [32] Community structure mining in big data social media networks with MapReduce
    Songchang Jin
    Wangqun Lin
    Hong Yin
    Shuqiang Yang
    Aiping Li
    Bo Deng
    [J]. Cluster Computing, 2015, 18 : 999 - 1010
  • [33] MapReduce-Based D_ELT Framework to Address the Challenges of Geospatial Big Data
    Jo, Junghee
    Lee, Kang-Woo
    [J]. ISPRS INTERNATIONAL JOURNAL OF GEO-INFORMATION, 2019, 8 (11)
  • [34] Scaling up MapReduce-based Big Data Processing on Multi-GPU systems
    Hai Jiang
    Yi Chen
    Zhi Qiao
    Tien-Hsiung Weng
    Kuan-Ching Li
    [J]. Cluster Computing, 2015, 18 : 369 - 383
  • [35] LandQυ2: A MapReduce-Based System for Processing Arable Land Quality Big Data
    Yao, Xiaochuang
    Mokbel, Mohamed E.
    Ye, Sijing
    Li, Guoqing
    Alarabi, Louai
    Eldawy, Ahmed
    Zhao, Zuliang
    Zhao, Long
    Zhu, Dehai
    [J]. ISPRS INTERNATIONAL JOURNAL OF GEO-INFORMATION, 2018, 7 (07)
  • [36] MapReduce-based large-scale online social network worm simulation
    He, Liang
    Feng, Deng-Guo
    Wang, Rui
    Su, Pu-Rui
    Ying, Ling-Yun
    [J]. He, L. (windhl@yahoo.cn), 1666, Chinese Academy of Sciences (24): : 1666 - 1682
  • [37] PFIMD: a parallel MapReduce-based algorithm for frequent itemset mining
    Mao Yimin
    Geng Junhao
    Deborah Simon Mwakapesa
    Yaser Ahangari Nanehkaran
    Zhang Chi
    Deng Xiaoheng
    Chen Zhigang
    [J]. Multimedia Systems, 2021, 27 : 709 - 722
  • [38] ScadiBino: An effective MapReduce-based association rule mining method
    Barkhordari, Mohammadhossein
    Niamanesh, Mahdi
    [J]. PROCEEDINGS OF THE SIXTEENTH INTERNATIONAL CONFERENCE ON ELECTRONIC COMMERCE (ICEC 2014), 2014, : 1 - 8
  • [39] MapReduce-Based network property verification technique for openFlow network
    Liu Y.
    Lei C.
    Zhang H.
    Yang Y.
    [J]. 1600, Science Press (53): : 2500 - 2511
  • [40] MapReduce-based Frequent Itemset Mining for Analysis of Electronic Evidence
    Jiang, Xueqing
    Sun, Guozi
    [J]. 2013 EIGHTH INTERNATIONAL WORKSHOP ON SYSTEMATIC APPROACHES TO DIGITAL FORENSIC ENGINEERING (SADFE), 2013,