Installing Apache Airflow

This section provides a step-by-step guide to installing Apache Airflow on the Azure Kubernetes Service cluster.

About this task

Apache Airflow is a platform to programmatically author, schedule, and monitor workflows. In Airflow, workflows are defined as code, and they are more maintainable, versionable, testable, and collaborative.

You can use Airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The Airflow scheduler executes tasks on an array of workers while following the specified dependencies.

Procedure

  1. Add Apache Airflow repositories to the Helm configuration.
  2. Download the Helm charts locally.
  3. Update the Airflow image with the HCL-provided Airflow image details in the Helm chart inside the values.yaml file.

    Edit the image repository and tag properties in the values.yaml file.

  4. Update the security context in the PostgreSQL values.yaml to disable seccompProfile.
  5. Deploy Apache Airflow using Helm.
  6. After deploying Apache Airflow, verify the deployed pods.