Handling Big Data Efficiently by using Map Reduce Technique

Extremely large amount of data is being captured by today's organizations and is continue to increase. It becomes computationally inefficient to analyze such huge data. Research in data mining has addressed problem in discovering knowledge from these continuously growing large data sets. The amount of raw data available has been increasing at an exponential rate. The valuable information is hidden in large databases. Data mining has become an interesting area to extract the embedded precious information from them. For many years it has been found its root in all kinds of application areas. Thus, gave evolution to many data mining methods which started to get applied in several real life fields. But not all the methods possess the capability to deal with and handle the huge collection of data. In recent years, numbers of computation and data intensive scientific data analyses are established. To perform the large scale data mining analyses so as to meet the scalability and performance requirements of big data, several efficient parallel and concurrent algorithms got applied. A lot of parallel algorithms are put into action using different parallelization techniques. Among them, some common techniques used are threads, MPI, MapReduce etc. which yield different performance and usability characteristics. In computing rigorous problems, the MPI model works efficiently. But it is a complicated task to bring this model into the practical use. There is currently considerable enthusiasm around the MapReduce paradigm for large-scale data analysis. It is inspired by functional programming which allows expressing distributed computations on massive amounts of data. It is designed for large-scale data processing as it allows to run on clusters of commodity hardware. A prominent parallel data processing tool MapReduce is gaining significant momentum from both industry and academia as the volume of data to analyze grows rapidly. In this paper, we are going to work around MapReduce, its advantages, disadvantages and how it can be used in integration with other technology.