Deploying CDP RTS

This section provides detailed instructions on how to deploy HCL CDP RTS using the Devtron in the OpenShift.

Prerequisites:

Make sure to create a rts secret with required data in the HashiCorp Vault before deploying the CDP RTS.

To create a rts secret in the HashiCorp Vault, follow the steps below:

  1. Create a rts secret sample key and value in the UI secret, and update ConfigMaps data with actual values.
    {
  "bootstrap.servers": "<KAFKA_HOST_1>:9096,<KAFKA_HOST_2>:9096",
  "security.protocol": "SASL_SSL",
  "sasl.jaas.config": "org.apache.kafka.common.security.scram.ScramLoginModule required username='<KAFKA_USER>' password='<KAFKA_PASSWORD>';",
  "sasl.mechanism": "SCRAM-SHA-512",

  "MysqlHost": "<DB_HOST>",
  "MysqlDb": "vrm",
  "MysqlUser": "<DB_USER>",
  "MysqlPass": "<DB_PASSWORD>",
  "MysqlPort": "3307",

  "KafkaConsumerConfig": "bootstrap.servers::<KAFKA_HOST>|security.protocol::SASL_SSL|sasl.jaas.config::org.apache.kafka.common.security.scram.ScramLoginModule required username='<KAFKA_USER>' password='<KAFKA_PASSWORD>';|sasl.mechanism::SCRAM-SHA-512|client.dns.lookup::use_all_dns_ips|group.id::k8s-rts-prod",

  "KafkaProducerConfig": "bootstrap.servers::<KAFKA_HOST>|security.protocol::SASL_SSL|sasl.jaas.config::org.apache.kafka.common.security.scram.ScramLoginModule required username='<KAFKA_USER>' password='<KAFKA_PASSWORD>';|sasl.mechanism::SCRAM-SHA-512|client.dns.lookup::use_all_dns_ips|acks::1|linger.ms::100",

  "MONGO_URI": "mongodb://<MONGO_USER>:<MONGO_PASSWORD>@<MONGO_HOST_1>:51090,<MONGO_HOST_2>:51090,<MONGO_HOST_3>:51090,<MONGO_HOST_4>:51090/segmentation?authSource=admin"
}

Deploying CDP RTS

To deploy the CDP RTS, follow the steps below:

  1. Navigate to the Devtron Chart Store, and select the cdp-rts chart to deploy.

  2. Now, configure and deploy the CDP RTS charts.

  3. In the YAML section, update the ConfigMap using below details, and deploy the charts.
    APP_INFRA: k8s
APP_REGION: ap-south-1
APP_TYPE: realtime
EXECUTOR_SERVICE_MAX_THREADS: "32"
EXECUTOR_SERVICE_MIN_THREADS: "16"
EXECUTOR_SERVICE_QUEUE_SIZE: "1000"
EXECUTOR_SERVICE_THREAD_TTL: "5"
FROM_EMAIL: hcl.com

KAFKA_CONSUMER_GROUP_ID: k8s-rts-prod
KAFKA_POLL_INTERVAL: "200"
KAFKA_PRODUCER_FAILURE_TOPIC: dmp_rt_failure
KAFKA_PRODUCER_META: "true"
KAFKA_PRODUCER_POOL_SIZE: "16"
KAFKA_PRODUCER_RAMANUJAN_TOPIC: dmp_rt_ramanujan
KAFKA_PRODUCER_TOPIC: dmp_rt_segments
KAFKA_SST_TOPIC: dmp_sst_rt_profile
KAFKA_DETECT_AGGREGATE_TOPIC: CDP_AGGREGATED_TRANS_TOPIC
KAFKA_TOPIC_CONSUMERS: 6,6,6
KAFKA_TOPIC_CONSUMER_BATCH_SIZE: 100,100,100
KAFKA_TOPIC_CONSUMER_DEFAULT_BATCH_TIMEOUT_MS: 200,200,200
KAFKA_TOPIC_CONSUMER_DEFAULT_BATCH_TIMEOUT_TRIGGER: 500,500,500
KAFKA_TOPIC_LIST: dmp_sst_rt_profile,CDP_QUALIFIED_TRANS_TOPIC,CDP_AGGREGATED_TRANS_TOPIC

LOG_LEVEL_APP: error
LOG_LEVEL_METRICS: off
LOG_LEVEL_ROOT: error
LOG_LEVEL_UTILS: off

MAX_BATCH_EXECUTE_RETRIES: "3"
MAX_EVENT_BATCH_LAG: "64"

MODEL_CONFIG_LOCAL_PATH: /disk1/rts/config/predictiveSegmentationModel.ini
MODEL_CONFIG_S3_PATH: s3://<masked-bucket>/sst/predictivesegments/rtson/predictiveSegmentationModel.ini

MODEL_CUTOFFS_LOCAL_PATH: /disk1/rts/config/predictiveSegmentationCutoffs.ini
MODEL_CUTOFFS_S3_PATH: s3://<masked-bucket>/sst/predictivesegments/rtson/predictiveSegmentationCutoffs.ini

MYSQL_REFRESH_FAILURE_TOLERANCE_COUNT: "3"
MYSQL_REFRESH_INTERVAL: "120000"

PMML_MODEL_CONFIG_LOCAL_PATH: /disk1/rts/config/predictivesegments/pmml_model_config/
PMML_MODEL_CONFIG_S3_PATH: s3://<masked-bucket>/sst/predictivesegments/pmml_model_config/rtson/

PMML_MODEL_DIRECTORY_LOCAL_PATH: /disk1/rts/config/predictivesegments/pmml_models/
PMML_MODEL_DIRECTORY_S3_PATH: s3://<masked-bucket>/sst/predictivesegments/pmml_models/rtson/

TO_EMAIL: ""

TSDB_HOST: <internal-service>.svc.cluster.local
TSDB_PORT: "80"

AWS_REGION: ap-south-1
SNS_ALERT_ENDPOINT: ""

FROM_EMAIL_ID: user@domain.com
ALERT_TO_EMAIL_ID: user@domain.com

BUCKET_TYPE: s3
awsRegion: us-south-1

emailType: SES
fromEmail: user@domain.com

smtpHost: <smtp-relay-host>
smtpPort: "20225"
toEmail: user@domain.com

KAFKA_DETECT_TOPIC: CDP_QUALIFIED_TRANS_TOPIC
KMS_IMPLEMENTATION: AWS

LOOKUP_KEY_IDENTIFIER: '[{"VIZVRM6525":"crmid"},{"VIZVRM6526":"crmid"},{"VIZVRM6527":"crmid"},{"VIZVRM6528":"crmid"},{"VIZVRM6529":"crmid"},{"VIZVRM6530":"crmid"},{"VIZVRM6531":"crmid"},{"VIZVRM6532":"crmid"},{"VIZVRM6533":"crmid"},{"VIZVRM6534":"crmid"},{"VIZVRM6535":"crmid"},{"VIZVRM6536":"crmid"},{"VIZVRM6537":"crmid"},{"VIZVRM6538":"crmid"},{"VIZVRM6539":"crmid"},{"VIZVRM6540":"crmid"},{"VIZVRM6541":"crmid"},{"VIZVRM6542":"crmid"},{"VIZVRM6543":"crmid"},{"VIZVRM6544":"crmid"},{"VIZVRM6545":"crmid"},{"VIZVRM6546":"crmid"},{"VIZVRM6547":"crmid"},{"VIZVRM6548":"crmid"},{"VIZVRM6549":"crmid"},{"VIZVRM6550":"crmid"}]'

  4. On successful deployment, validate the deployment as shown below.