Big Data Analysis: Implementations of Hadoop Map Reduce, Yarn and Spark

Abstract

Nowadays, with the increasingly important role of technology, the internet and huge size of data, it has become not only possible, but necessary for management and analyzing these data, where it is difficult to process and retrieve information related to that data. Moreover, the amount of memory consumed by such data reached to terabytes or petabytes, which make it difficult for processing, analyzed, and retrieving. Also, many techniques have been carried to process big data. The dealing with the statistical programs became very hard. There are a number of algorithms that is used in big data processing, such as Mapreduce. Many obstructions and challenges face the big data processing as: poor bounded-time performance in heavy activities and high-priced cost. In this study, different big data implementations are demonstrated, also, we propose open issues and challenges raised on big data implementations. The findings compares several big data platforms which are; Hadoop, Yarn and Spark. Finally, we provide useful recommendations for further research about the best one between these implementations to process the data according to specific bases.

Keywords: Big data, Mapreduce, Hadoop, Spark, Yarn.