Installing Spark

This section provides a step-by-step guide to installing Spark

About this task

Apache Spark is an open-source unified analytic engine for large-scale data processing. Spark provides an interface for programming clusters with implicit data parallelism and fault tolerance.

Procedure

  1. Add bitnami Repositories to helm configuration. 
    helm repo add bitnami https://charts.bitnami.com/bitnami
helm repo update
  2. Deploy Spark in a namespace using Helm. 
    helm install my-spark bitnami/spark --namespace spark
  3. On successful deployment, verify that all the components are running correctly. 
    oc get pods -n spark
  4. As a result, the output will be as shown below. 
    my-spark-master-0 1/1 Running 0 2d4h
my-spark-worker-0 1/1 Running 0 2d4h
my-spark-worker-1 1/1 Running 0 2d4h