A Mathematical Appraisal: Evolution of Distributed File System and Hadoop

Abstract

The  fast  growing  technology   has  left  a  great   impact  on  the  human  life. Many traditional systems are either replaced or running in parallel with their electronic counterpart. As for example: - the traditional postal system is now nearly replaced by mobile phones and emails. The electronic system is providing more functionalities than their traditional counterparts. Due to social media, peoples may communicate with each other, share their thoughts and moments of life in form of texts, images or videos. On the  other  hand,  to enhance  technologies  and  knowledge many research  activities  are propelled  and data from  different sources  are  gathered in large  volume for further  analysis. In short today’s world is surrounded with large volume of data in different form. This put a requirement for effective management of these billions of terabytes of electronic data generally called BIG DATA.  The effective management must be based on proven mathematical concepts so that chance of casualties may be reduced.

This  paper  presents  a mathematical  appraisal for evolution   of  distribution of  file data   and  explains some basic  solution  of primitive  problems  based  on probability theory.

Keywords: BIG DATA,   Distributed System, DFS, Commodity Hardware, Hadoop, Hadoop File System, HDF.