Big Data Processing with Apache Spark in Tertiary Institutions: Spark Streaming

Emmanuel Boachie, Chunlin Li

Abstract


In tertiary institutions, different set of information are derived from the various department and other functional sections. Individual departments and other functional sections in the institutions manage their data separately. This situation has resulted in huge number of different set of data across the various departments in tertiary institutions. There is no centralized data centre where data/information can be retrieved for the management committee when the need arises. In academic institution data captured is restricted to the institution which collected it but centralisation of the various data in the various functional sections does not exist. This makes it difficult for the management committee to take decisions based on relevant information needed. In order to address this problem, we proposed Spark Streaming. Spark Streaming is an element which facilitates processing of live flows of data. Spark streaming will able to capture data in real time, process it and make it available to the management committee when the need arises

Keywords: Spark, Streaming, Big data, Processing, Tertiary, Institution


Full Text: PDF
Download the IISTE publication guideline!

To list your conference here. Please contact the administrator of this platform.

Paper submission email: JIEA@iiste.org
ISSN (Paper)2224-5782 ISSN (Online)2225-0506
Please add our address "contact@iiste.org" into your email contact list.
This journal follows ISO 9001 management standard and licensed under a Creative Commons Attribution 3.0 License.
Copyright © www.iiste.org