Installing Nifi

This section provides a step-by-step guide to installing Nifi.

To install Nifi, follow the steps below:

Mount an external file system disk to a VM at /data.
```
mount external file system disk to the VM at /data
```

Perform docker installation with below commands.

yum update -y
yum install -y yum-utils
yum-config-manager --add-repo https://download.docker.com/linux/rhel/docker-ce.repo
yum install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin --allowerasing
wget https://github.com/opencontainers/runc/releases/download/v1.1.12/runc.amd64 -O /usr/local/sbin/runc
cp /usr/local/sbin/runc /usr/bin/runc
systemctl start docker
curl -L "https://github.com/docker/compose/releases/download/v2.12.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
mv /usr/local/bin/docker-compose /usr/bin/docker-compose
chmod +x /usr/bin/docker-compose
systemctl status docker
ln -s /data /disk1
mkdir -p /disk1/nifi
mkdir -p /disk1/nifi/data
mkdir -p /disk1/nifi/conf
mkdir -p /disk1/nifi/logs

Download the postgresql-42.6.0.jar file and copy into /disk1/nifi. cd /disk1/nifi; wget https://jdbc.postgresql.org/download/postgresql-42.6.0.jar

Running NiFi in Docker with SSL Enabled

Create Self-Signed Certificate

To create a self-signed certificate using Apache NiFi Toolkit, follow the steps below:

Download the latest Apache NiFi Toolkit from the official website, and extract the archived file.
```
unzip nifi-toolkit-1.24.0-bin.zip
```
Change the terminal directory to the toolkit folder.
```
cd nifi-toolkit-1.24.0-bin
```

Generate the SSL certificates and the necessary configurations using the following command.

./bin/tls-toolkit.sh standalone -n <VM_IP> -C 'CN=admin,OU=NiFi' --subjectAlternativeNames '<VM_IP>,nifi00,<VM_Hostname>,localhost,0.0.0.0'

As a result, the above command generates the certificate, key, keystore, truststore, and the properties file for the NiFi server deployed in the local host.
```
├── l<VM_IP>
│   ├── keystore.jks
│   ├── nifi.properties
│   └── truststore.jks
├── nifi-cert.pem
├── nifi-key.key
└── CN=admin_OU=NiFi.p12
```
Note: The Subject Alternative Names have multiple server names. These names will be the hostnames of our hardcoded Apache NiFi containers. Make sure to add as many hostnames as possible based on your cluster size.

Create a NiFi Cluster using a docker compose file

Create a new folder as "nifi".
```
mkdir ~/nifi
cd ~/nifi
```

Copy the keystore.jks, truststore.jks, nifi-cert.pem, and nifi-key.key into this folder using the commands.

cp $NIFI_TOOLKIT_HOME/VM_IP/keystore.jks ./
cp $NIFI_TOOLKIT_HOME/VM_IP/truststore.jks ./
cp $NIFI_TOOLKIT_HOME/VM_IP/nifi-cert.pem ./
cp $NIFI_TOOLKIT_HOME/VM_IP/nifi-key.key ./

Create a new file as docker-compose.yaml and use the following content for the yaml file.

version: "3"
services:
  zookeeper:
    hostname: zookeeper
    container_name: zookeeper
    image: bitnami/zookeeper:3.9.1
    restart: always
    environment:
      - ALLOW_ANONYMOUS_LOGIN=yes
    networks:
      - nifinet

  nifi00:
    image: apache/nifi:1.24.0
    container_name: nifi00
    hostname: rmmyclddl334821.nonprod.hclpnp.com
    restart: always
    ports:
      - 8443:8443
    depends_on:
      - zookeeper
    volumes:
      - /disk1/nifi/custom_processors:/opt/nifi/nifi-current/extensions
      - /disk1/nifi/postgresql-42.6.0.jar:/opt/nifi/nifi-current/lib/postgresql-42.6.0.jar
      - nifi-conf:/opt/nifi/nifi-current/conf
      - nifi-logs:/opt/nifi/nifi-current/logs
      - ./keystore.jks:/opt/certs/keystore.jks
      - ./truststore.jks:/opt/certs/truststore.jks
    networks:
      - nifinet
    environment:
      - NIFI_WEB_HTTPS_PORT=8443
      - SINGLE_USER_CREDENTIALS_USERNAME=admin
      - SINGLE_USER_CREDENTIALS_PASSWORD=ctsBtRBKHRAx69EqUghvvgEvjnaLjFEB
      - NIFI_WEB_PROXY_HOST=
      - NIFI_WEB_HTTPS_HOST=
      - NIFI_CLUSTER_ADDRESS=
      - NIFI_REMOTE_INPUT_HOST=
      - AUTH=tls
      - KEYSTORE_PATH=/opt/certs/keystore.jks
      - KEYSTORE_TYPE=JKS
      - KEYSTORE_PASSWORD=M5ZuMixds5wSWEEFku0uYI7FemY8gn9CfL8OEq9YtO8
      - TRUSTSTORE_PATH=/opt/certs/truststore.jks
      - TRUSTSTORE_TYPE=JKS
      - TRUSTSTORE_PASSWORD=5DNRwzx46oO1ue/PV9JSbPlf/CpHcHcIn4RZCjZCbp8
      - NIFI_SECURITY_USER_AUTHORIZER=single-user-authorizer
      - NIFI_SECURITY_USER_LOGIN_IDENTITY_PROVIDER=single-user-provider
      - NIFI_CLUSTER_NODE_PROTOCOL_PORT=8082
      - NIFI_ZK_CONNECT_STRING=zookeeper:2181
      - NIFI_ELECTION_MAX_WAIT=1 min
      - NIFI_CLUSTER_IS_NODE=true
      - NIFI_SENSITIVE_PROPS_KEY=bf4xSLVSAmtex/qtcP5uMTbCrxaP+8q5WjELaYXTkkQ=
      - JVM_ARGS=-XX:MaxDirectMemorySize=2GB -Xms1g -Xmx2g
networks:
  nifinet:
    driver: bridge
volumes:
  nifi-data:
    driver: local
    driver_opts:
      type: none
      o: bind
      device: /disk1/nifi/data

  nifi-conf:
    driver: local
    driver_opts:
      type: none
      o: bind
      device: /disk1/nifi/conf

  nifi-logs:
    driver: local
    driver_opts:
      type: none
      o: bind
      device: /disk1/nifi/logs

Note: The NiFi server is configured with a single username (admin) and password. Running the described configuration will launch a NiFi container using the keystore and truststore files generated earlier. Make sure the hostname matches the subjectAlternativeNames specified during the setup process and each container requires a unique hostname.

In the above configuration, update the KEYSTORE_PASSWORD and TRUSTSTORE_PASSWORD with the respective value of nifi.security.keystorePasswd and nifi.security.truststorePasswd that are available in the nifi.properties file generated earlier.

Start NiFi in Docker

To start the NiFi in docker, follow the steps below:

In the nifi folder, open the terminal and run the following command.
```
docker-compose up -d
```
Note: Make sure that the Docker Compose is installed.
Wait till the NiFi to boot up, and then visit https://<VM_Hostname>:8443/nifi from your browser.
In the landing page, a warning message is displayed. Accept the risk and visit the page.
Enter user credentials defined in the docker-compose.yaml to login.
In the hamburger menu, select the "Cluster" option. As a result, the page displays the NiFi containers defined in the docker-compose.yml as the members of the NiFi cluster.