Research Article Open Access

Novel Apache Spark based Algorithm to Solve Dirichlet Problem for Poisson Equation in 3D Computational Domain

Shomanov Aday1 and Mansurova Madina1
  • 1 al-Farabi Kazakh National University, Kazakhstan

Abstract

Parallel computations are essential tool in solving large-scale computationally demanding problems. Due to large diversity and heterogeneity of the currently available parallel processing techniques and paradigms it is usually difficult to find the right solution that will perform well according to every performance metric. As one of the recent developments in parallel computing Apache Spark framework allows to process petabyte-scale data and possesses properties such as fault tolerance, scalability, load balancing and mechanisms of in memory computations across nodes of the cluster. All of these features are attractive for high performance scientific computing. It has been shown that Apache Spark outperforms Hadoop implementation of some machine learning algorithms by orders of magnitude. Since Hadoop platform is not well suited for iterative computing, typical for many computational problems, in this study we investigate performance characteristics of Apache Spark on scientific computing problems, particularly for solving Dirichlet problem for Poisson's equation. An algorithm for solving Dirichlet problem for Poisson's equation is described and analyzed and compared to optimized Hadoop-based implementations. Apache Spark uses new distributed data structure called RDD. Presented algorithm consists of operations on RDD such as mapping, grouping and partitioning. The benefits and drawbacks of the algorithm as well as applicability for stencil type computations are discussed and analyzed.

Journal of Computer Science
Volume 12 No. 10, 2016, 502-509

DOI: https://doi.org/10.3844/jcssp.2016.502.509

Submitted On: 9 August 2016 Published On: 15 December 2016

How to Cite: Aday, S. & Madina, M. (2016). Novel Apache Spark based Algorithm to Solve Dirichlet Problem for Poisson Equation in 3D Computational Domain. Journal of Computer Science, 12(10), 502-509. https://doi.org/10.3844/jcssp.2016.502.509

  • 3,307 Views
  • 2,209 Downloads
  • 0 Citations

Download

Keywords

  • Hadoop
  • Spark
  • RDD
  • HPC