Deploying Kafka UI for AMQ Streams

This section provides a comprehensive guide for deploying Kafka UI

Kafka UI, a web-based interface for managing and monitoring Apache Kafka clusters deployed through AMQ Streams. Kafka UI simplifies interactions with Kafka by providing an intuitive dashboard for exploring topics, consumers, and producers, as well as monitoring real-time cluster performance.

The page covers creating secrets for secure authentication, configuring necessary services and deployments, and integrating the UI with the Kafka cluster. By following this guide, you will enable seamless access to Kafka's operational insights, facilitating efficient cluster management and troubleshooting.

Prerequisites

Make sure the following things are in place, before creating the kafka-ui-secrets.yaml file:

Convert bootstrap servers URL to base-64.
```
<Your-cluster-name>-kafka-bootstrap.cdp.svc:9092
```
Convert password kafka-ui-passwd to base-64.

Convert the sasl-jass-config value to base-64.

org.apache.kafka.common.security.scram.ScramLoginModule required username="kafka-user-used-in-kafka-cdp-secret" password="base64 password";

Convert sasl-mechanism SCRAM-SHA-512 to base-64.
Convert security-protocol SASL_PLAINTEXT to base-64

Deploy Kafka UI

To deploy the Kafka UI, follow the steps below:

Update the converted values in the below yaml file.

apiVersion: v1
data:
  bootstrap-servers: aGNsc3ctc2JpLWRldi1rYWZrYS1rYWZrYS1ib290c3RyYXAuY2RwLnN2Yzo5MDky
  password: #pasaword#
  sasl-jaas-config: b3JnLmF..=
  sasl-mechanism: U0NSQU0tU0hBLTUxMg==
  security-protocol: U0FTTF9TU0w=
kind: Secret
metadata:
  name: kafka-ui-secret
type: Opaque

Create the Kafka UI Secret resource in the specified namespace.
```
oc apply -f kafka-ui-secret.yaml -n <your namespace>
```

Create a service definition in kafka-ui-service.yaml to expose the Kafka UI within the cluster.

apiVersion: v1
kind: Service
metadata:
  name: kafka-ui-service
spec:
  ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: 8080
  selector:
    app: kafka-ui

Deploy the Kakfa UI service.

oc apply -f kafka-ui-service.yaml -n <your namespace>

Define the Kafka UI deployment in kafka-ui-deployment.yaml, including environment variables and secrets.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: kafka-ui
spec:
  replicas: 1
  selector:
    matchLabels:
      app: kafka-ui
  template:
    metadata:
      labels:
        app: kafka-ui
    spec:
      containers:
        - env:
            - name: KAFKA_CLUSTERS_0_NAME
              value: kafka-cluster
            - name: KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS
              valueFrom:
                secretKeyRef:
                  key: bootstrap-servers
                  name: kafka-ui-secret
            - name: KAFKA_CLUSTERS_0_PROPERTIES_SECURITY_PROTOCOL
              valueFrom:
                secretKeyRef:
                  key: security-protocol
                  name: kafka-ui-secret
            - name: KAFKA_CLUSTERS_0_PROPERTIES_SASL_MECHANISM
              valueFrom:
                secretKeyRef:
                  key: sasl-mechanism
                  name: kafka-ui-secret
            - name: KAFKA_CLUSTERS_0_PROPERTIES_SASL_JAAS_CONFIG
              valueFrom:
                secretKeyRef:
                  key: sasl-jaas-config
                  name: kafka-ui-secret
            - name: KAFKA_CLUSTERS_0_METRICS_PORT
              value: "11001"
            - name: DYNAMIC_CONFIG_ENABLED
              value: "true"
            - name: AUTH_TYPE
              value: LOGIN_FORM
            - name: SPRING_SECURITY_USER_NAME
              value: admin
            - name: SPRING_SECURITY_USER_PASSWORD
              valueFrom:
                secretKeyRef:
                  key: password
                  name: kafka-ui-secret
          image: provectuslabs/kafka-ui:latest
          name: kafka-ui
          ports:
            - containerPort: 8080

Deploy kafka UI in the namespace.

oc apply -f kafka-ui-deployment.yaml -n <your namespace>

On successful deployment, expose the Kafka UI service to create a route and access the Kafka UI.

oc expose svc kafka-ui-service -n <your namespace>
oc get routes -n <your namespace>
Login username : - admin
Login password : - kafka-ui-passwd

Use the exposed route URL to access Kafka UI in your browser.
Login credentials:
- Username:admin
- Password: kafka-ui-passwd
Upon successful deployment, you can use Kafka UI to manage and monitor your Kafka clusters effectively.