Research Article Open Access

A Systematic Literature Review for Implementing Data Ops in the Data Warehouse Lifecycle during the ETL Phase

Ahmed Bahaa1, Sherif Ragab Eldemerdash2 and Hanan Fahmy3
  • 1 Helwan University, Egypt
  • 2 Faculty of Computers and AI Beni-suef University, Egypt
  • 3 Faculty of Computers and AI Helwan University, Egypt

Abstract

Nowadays, no one can deny the importance of Data Ware House (DWH) in all organizations. The most important components in Data Ware House (DWH) are the Extraction, Transformation, Loading (ETL) phase. Data cleaning is a basic piece of the transformation stage in Data Warehousing. This may affect critical activities such as data collection and decision-making in various organizations Data Ops is an evaluation technique of Dev Ops in the data domain. This study conducts a Systematic Literature Review (SLR) to assess the previous studies of data warehouses related to Data Ops efforts. This study collects 55 primary studies related to the detection of Data Scrubbing, Data Consistency, Data warehouse, Dev Ops and Data Ops and we have conducted a Systematic Literature Review (SLR). Based on these findings, we discuss many concerns related to the study of current approaches in terms of abstraction level, metrics used, implementation and validation. That is why the analysis covers the published efforts between 2016 and 2021 since Data Ops is a significantly new technique. The survey should cover only research that took plan in recent years. The result of the study observed that 29% of the studies focused on solving the importance of data quality in the data warehouse, 62% of them focused on related Dev Ops, only 9% focused on Data Ops techniques and no 0% survey on enhancing ETL phase with Data Ops. This SLR brings to the attention of the research community several opportunities for using Data Ops in future research and the nearly proposed model DW Ops.

Journal of Computer Science
Volume 17 No. 11, 2021, 1011-1030

DOI: https://doi.org/10.3844/jcssp.2021.1011.1030

Submitted On: 23 June 2021 Published On: 10 November 2021

How to Cite: Bahaa, A., Eldemerdash, S. R. & Fahmy, H. (2021). A Systematic Literature Review for Implementing Data Ops in the Data Warehouse Lifecycle during the ETL Phase. Journal of Computer Science, 17(11), 1011-1030. https://doi.org/10.3844/jcssp.2021.1011.1030

  • 2,272 Views
  • 1,208 Downloads
  • 0 Citations

Download

Keywords

  • Data Scrubbing
  • Data Quality
  • Data Ware House (DWH)
  • Dev Ops
  • Data Ops
  • ETL
  • Data Transformation