A Survey on Internet Traffic Measurement and Analysis

Abstract

As the number of Internet users increasing rapidly in this world, Internet traffic is also increased. In computer network traffic measurement is the process of measuring the amount and type of traffic on a particular network. Internet traffic measurement and analysis are mostly used to characterize and analysis of network usage and user behaviour, but faces the problem of scalability under the explosive growth of Internet traffic and high speed access. It is not easy to handle Tera and Pera-byte traffic data with single server. Scalable Internet traffic measurement and analysis is difficult because a large dataset requires matching commutating and storage resources. To analyse this traffic multiple tools are available. But they do not perform well when the traffic data size increase. As data grows it is necessary to increase the necessary infrastructure to process it. The distributed File System can be used for this purpose, but it has certain limitation such as scalability, availability and fault tolerant. Hadoop is popular parallel processing framework that is widely used for working with large datasets and it is an open source distributed computing platform having MapReduce for distributed processing and HDFS to store huge amount of data. In future work we will present a Hadoop-based traffic monitoring system that perform a multiple types of analysis on large amount of internet traffic in a scalable manner

Keywords- Traffic monitoring, Hadoop, MapReduce, HDFS, NetFlow.