Application of big data in network security

Network security incidents show explosive growth, not only in people's daily life has had a huge impact in the network intrusion behavior, and even bring harm to the security of the country. How to find and deal with the abnormal traffic is a very meaningful topic. This paper discusses the application of big data technology in network security analysis from the aspects of security data collection, query, retrieval and mining. With Flume, HDFS, Hive as the basic framework, this paper designs a kind of abnormal flow large data analysis platform, respectively, the data collection, storage, analysis, etc.. The application of big data technology in network security analysis can not only reduce the storage of the network information data, but also improve the storage capacity of the database. Big data analysis technology will be more and more applied in safety monitoring, the use of big data security analysis method can not only solve the collection and storage of massive data, using data mining method can be more proactive and flexible to deal with new complex irregularities and unknown changeful risk.