Big Data Analytics for Network Congestion Management Using Flow-Based Analysis

Due to explosive growth of traffic volume, it is hard to accumulate Internet traffic on a single machine. In this paper, a Hadoop-based traffic analysis system accepts input from multiple data traces. Hadoop facilitates scalable data processing and storage services on a distributed computing system. This system accepts input of large scales of trace file generated from traffic measurement tool like Wireshark-identifies flows running on the network from this trace file. Characteristics of flow describe the pattern of network traffic; it helps network operator understand network capacity planning, traffic engineering, and fault handling. The main objective is to design and implement a traffic flow identification system using Hadoop. The traffic flow identification system will be very useful for network administrator to monitor faults and also to plan for the future.