Research Article Open Access

Real Time Anomaly Detection in Massive Data Streams with ELK Stack

Jakup Fondaj1 and Zirije Hasani2
  • 1 South East European University, Macedonia
  • 2 University of Prizren, Kosovo
Journal of Computer Science
Volume 15 No. 6, 2019, 814-823

DOI: https://doi.org/10.3844/jcssp.2019.814.823

Submitted On: 13 January 2019 Published On: 17 June 2019

How to Cite: Fondaj, J. & Hasani, Z. (2019). Real Time Anomaly Detection in Massive Data Streams with ELK Stack. Journal of Computer Science, 15(6), 814-823. https://doi.org/10.3844/jcssp.2019.814.823

Abstract

Real time anomaly detection is very popular topic nowadays this because the number of data generated every day is larger and larger. Facing with the phenomena of Big Data is not an easy task. The main aim of this research is to fine appropriate architecture for real-time big data analytic and its main task is to detect anomalies in this real-time data. In this paper we show the implementation of anomaly detection algorithm in real time infrastructure in order to find anomalies as soon as possible. We have proposed architecture for real time anomaly detection by adding some new components and the main part of the infrastructure is Timelion which enable implementation of different algorithms for anomaly detection. The research is focused to develop infrastructure to monitor e-dnevnik (education national system in Macedonia) application server and to detect errors in order to scale up the performance.

  • 798 Views
  • 1,069 Downloads
  • 0 Citations

Download

Keywords

  • Real Time
  • Big Data
  • Timelion
  • Infrastructure for Anomaly Detection