Deploying HCL CDP ETA Events

This section provides detailed instructions on how to deploy HCL CDP Etaevents using the Devtron in the AWS.

Prerequisites

Make sure to create a cdp-etaevents secret in the AWS secret manager before deploying CDP ETA Events.

To create a CDP Etaevents secret in the AWS secret manager , follow the steps below:

  1. Create a cdp-etaevents secret sample key and value, and update the actual values as shown below.
    {
  "KafkaConsumerConfig": "",
  "KafkaProducerConfig": "",
  "REDIS_SERVER_IP": "",
  "REDIS_SERVER_PORT": "6379",
  "KAFKA_PRODUCER_HOSTNAME": "etaeventshclhost",
  "TSDB_HOST": "",
  "TSDB_PORT": "80",
  "KAFKA_TOPIC_CONSUMER_DEFAULT_BATCH_TIMEOUT_MS": "1000,1000,1000",
  "KAFKA_TOPIC_CONSUMER_DEFAULT_BATCHTIMEOUT_TRIGGER": "5000,5000,5000",
  "MIN_KAFKA_MESSAGE_SEGMENTS": "CeleryETALem::4|ETACallback::2",
  "KAFKA_MESSAGE_TIMESTAMP_INDEX": "CeleryETALem::1|ETACallback::0",
  "KafkaProducerConfigaps1": ""
}
  2. Update ConfigMaps data with actual values, and create required Kafka topics before deployment.

Deploy CDP Etaevents

To deploy the cdp-etaevents using Devtron in the AWS platform, follow the steps below:

  1. Navigate to the Devtron Chart Store, and select the cdp-etaevents chart to deploy.

  2. Now, configure and deploy the cdp-etaevents charts.

  3. In the YAML section, update the ConfigMap using the below details and deploy the chart.
    APP_LOG_LEVEL: debug
ROOT_LOG_LEVEL: WARN
LOG_LEVEL_UTILS: debug
KAFKA_CONSUMER_GROUP: EtaEventsConsumerGroupLemAzure
KAFKA_CONSUMER_PREFIX: DelayedEventsConsumerLemAzure
KAFKA_TOPIC_LIST: CeleryETALem,DeleteETALem,ETACallback
KAFKA_TOPIC_CONSUMERS: 3,3,3
KAFKA_BATCH_SIZE: 1,1,1
ETANotify: ETANotify
TRIGGER: ETANotify
JOURNEYBUILDER: ETANotifyJB
TsdbEventsSent: TsdbEventsSent
KAFKA_PRODUCER_META: "false"
TsdbEventsRcvdMeanLatency: TsdbEventsRcvdMeanLatency
TsdbEventsRcvdLatency: TsdbEventsRcvdLatency
TsdbEventsRcvd: TsdbEventsRcvd
KAFKA_EXECUTOR_SERVICE_MIN_THREADS: "10"
KAFKA_EXECUTOR_SERVICE_MAX_THREADS: "10"
KAFKA_EXECUTOR_SERVICE_THREAD_TTL: "20"
KAFKA_EXECUTOR_SERVICE_QUEUSE_SIZE: "1000"
BATCH_SIZE: "1"
BATCH_TIME_THRESHOLD: "2"
CELERY_ETA: CeleryETALem
DELETE_ETA: DeleteETALem
ETA_CALLBACK: ETACallback
KAFKA_PRODUCER_ID: DelayedEventProducerLemAzure
KAFKA_SERIALIZER_CLASS: kafka.serializer.StringEncoder
KAFKA_REQUIRED_ACKS: "1"
KAFKA_REQUEST_TIMEOUT: "10000"
KAFKA_PRODUCER_TYPE: async
KAFKA_COMPRESSION_CODEC: gzip
KAFKA_MAX_RETRIES: "3"
KAFKA_RETRY_BACKOFF: "1000"
KAFKA_PRODUCER_BATCHING_SIZE: "500"
KAFKA_PRODUCER_BUFFER_TIME: "1000"
KAFKA_PRODUCER_Q_ENQ_TIMEOUT: "0"
KAFKA_PRODUCER_HOSTNAME: b-1.hclswhsshxcbdevmsk.us1d78.c4.kafka.ap-south-1.amazonaws.com:9096,b-2.hclswhsshxcbdevmsk.us1d78.c4.kafka.ap-south-1.amazonaws.com:9096
PRODUCER_BATCH_SIZE: "200"
PRODUCER_BATCH_TIME_THRESHOLD: "2"
PRODUCER_POOL_SIZE: "10"
EVENT_EXECUTOR_SERVICE_MIN_THREADS: "10"
EVENT_EXECUTOR_SERVICE_MAX_THREADS: "20"
EVENT_EXECUTOR_SERVICE_THREAD_TTL: "20"
EVENT_EXECUTOR_SERVICE_QUEUE_SIZE: "1000"
REDIS_CONNECTION_TIMEOUT: "2000"
TTL_MULTIPLIER: "1.2"
IS_METRIC_LOGGER_ACTIVE: "true”

  4. On successful deployment, validate the deployment as shown below.