Configuring remote cache invalidation
Using Apache Kafka and Apache ZooKeeper, you can run cache invalidation from the HCL Commerce Transaction server, or remotely from a Liberty server. You can enable and customize this feature by editing HCL Commerce configuration files.
Before you begin
crsApp:
enabled: true
name: crs-app
image: commerce/crs-app
tag: v9-latest
replica: 1
resources:
requests:
memory: 2048Mi
cpu: 500m
limits:
memory: 4096Mi
cpu: 2
## when using custom envParameters, use key: value format
envParameters:
auth:
ZOOKEEPER_SERVERS: my-kafka-zookeeper.kafka.svc.cluster.local:2181
KAFKA_SERVERS: my-kafka.kafka.svc.cluster.local:9092
KAFKA_TOPIC_PREFIX: sample
live:
ZOOKEEPER_SERVERS: my-kafka-zookeeper.kafka.svc.cluster.local:2181
KAFKA_SERVERS: my-kafka.kafka.svc.cluster.local:9092
KAFKA_TOPIC_PREFIX: sample
nodeLabel: ""
fileBeatConfigMap: ""
nodeSelector: {}
coresSharingPersistentVolumeClaim: ""
- KAFKA_SERVERS=<kafkaServerHostOrIPList>
- KAFKA_TOPIC_PREFIX=<kafkaTopicPrefix>
- KAFKA_AUTHENTICATION_USERID=<kafkaAuthenticationUserID>
- KAFKA_AUTHENTICATION_PASSWORD=<kafkaAuthenticationPassword>
run set-kafka-server
<KafkaServers> <TopicPrefix> <ZookeeperServers>"https://help.hcltechsw.com/commerce/9.0.0/developer/refs/rre_transaction.html
If the <KafkaServers> string starts with
(no-crs)
, then the Transaction server will
not publish invalidation messages intended to be received by the Store server.
DB2
and
TS-App
, are in separate timezones while setting Kafka/
Zookeeper in Version 9, invalidations may not occur correctly. In the Kafka
properties ensure that log.message.timestamp.type =
LogAppendTime
. By default this value is set to
log.message.timestamp.type =
CreateTime
.Kafka authentification password should be the encrypted
string using the wcs_encrypt password
. The ASCII encrypted
string should be saved in the configuration file, along with the password for
the Kafka servers' Sasl.
About this task
Procedure
- Open WebSphere Commerce Developer and switch to the Enterprise Explorer view.
-
Go to the following directory and open the Transaction server custom
configuration file wc-component.xml for editing.
workspace_dir\WC\xml\config\com.ibm.commerce.foundation-ext.
If the directory or configuration file do not exist, create them.
-
Locate the wc.store.remote.kafka parameter of the
RemoteStoreConfiguration
configuration grouping. Add the addresses of the Apache Kafka broker clusters, as a comma-separated string, to the value attribute. Set the port numbers according to your local environment. For example:<_config:configgrouping name="RemoteStoreConfiguration"> ... <!-- value to kafka servers connection string --> <_config:property name="wc.store.remote.kafka" value="kafka-broker1:9092,kafka-broker2:9092,kafka-broker3:9092"/> ... </_config:configgrouping>
-
Locate the wc.store.remote.kafka.topicPrefix parameter in the same
configuration grouping. Add the topic prefix for cache invalidation. This string contains the same
value as the prefix configured in the remote store server. The value must be same across Transaction
servers.
<!-- value to kafka servers topic prefix --> <_config:property name="wc.store.remote.kafka.topicPrefix" value="sampleprefix"/>
-
Locate the wc.remote.zookeeper parameter of the
TransactionKafkaConfiguration
configuration grouping. Add the addresses of the Apache ZooKeeper servers, as a comma-separated string, to the value attribute. Set the port numbers according to your local environment. For example:<_config:configgrouping name="TransactionKafkaConfiguration"> ... <_config:property name="wc.remote.zookeeper" value="zookeeper1:2181,zookeeper2:2181,zookeeper3:2181/kafka"/> ... </_config:configgrouping>
- Save and close the custom configuration file.
-
Go to the following directory.
workspace_dir\Liberty\servers\crsServer\configDropins\overrides
- Edit the custom configuration file jndi.xml. If the file does not exist, create it.
-
Add the configuration string for the ZooKeeper servers to the
jndiName attribute of the
jndiEntry
element. The value is the same as the ZooKeeper configuration string in the Transaction server.<jndiEntry jndiName="com.ibm.commerce.foundation.server.services.zookeeper.hostnameport" value=""/>
-
Add the topic prefix string. This value is the same as the topic prefix string
in the Transaction server.
<jndiEntry jndiName="com.ibm.commerce.foundation.server.services.cacheinvalidation.topicprefix" value=" "/>
- Save and close the custom configuration file.
- Deploy your changes to the HCL Commerce runtime environment.
- Read up on proven approaches
- Review Is your Zookeeper in need of maintenance? on the HCL blog, and the Apache Kafka documentationfor tips and utilities that can help you optimize your configuration.
- Configure message retention
- Default message retention is 7 days, which is very excessive when messages are no longer needed after cache invalidations have been processed by all applications. A retention time of 10 minutes is sufficient for most configurations.
- Disable automatic topic setup
- By default, if a message is sent to a topic that is not set up already, Kafka will automatically create it. This approach results in configurations such as message retention and replication-factor not being set up, which can cause outage issues. To avoid such issues, disable the automatic configuration of topics. This forces the client to set up topics with the parameters you have defined.
- Use replicas
- Replicas provide for high availability. For example, with two
brokers the configuration can be as
follows:
CacheInvalidation leader: broker0 replica: broker1 PeerCacheInvalidation leader: broker1 replica: broker0
- Optimize Producer configurations
- Use the following settings with the
Producers:
For more information, see 3.3 Producer Configs in the Apache Kafka documentation.//acks=all, This means the leader will wait for the full set of in-sync replicas to acknowledge the record. //This guarantees that the record will not be lost as long as at least one in-sync replica remains alive. configValues.put(ProducerConfig.ACKS_CONFIG, "all"); // Setting a value greater than zero will cause the client to resend any record whose send fails with a potentially transient error. configValues.put(ProducerConfig.RETRIES_CONFIG, 0); //The producer will attempt to batch records together into fewer requests whenever multiple records are being sent to the same partition. configValues.put(ProducerConfig.BATCH_SIZE_CONFIG, 16384); //LINGER_MS_CONFIG, This setting gives the upper bound on the delay for batching. configValues.put(ProducerConfig.LINGER_MS_CONFIG, 1); //buffer.memory, The total bytes of memory the producer can use to buffer records waiting to be sent to the server. configValues.put(ProducerConfig.BUFFER_MEMORY_CONFIG, 33554432);
- Optimize Consumer configurations
- Use the following settings with the
Consumers:
For more information, see 3.4 Consumer Configs in the Apache Kafka documentation.//ENABLE_AUTO_COMMIT_CONFIG : If true, periodically commit to Kafka the offsets of messages already returned by the consumer. configValues.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, "true"); // AUTO_OFFSET_RESET_CONFIG : earliest: automatically reset the offset to the earliest offset configValues.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest");
Example
What to do next
- DistributedMaps in the Transaction server
-
- WCSessionDistributedMapCache
- WCCatalogEntryDistributedMapCache
- WCCatalogGroupDistributedMapCache
- WCMiscDistributedMapCache
- WCUserDistributedMapCache
- WCPriceDistributedMapCache
- WCMarketingDistributedMapCache
- WCPromotionDistributedMapCache
- WCContractDistributedMapCache
- WCTelesalesDistributedMapCache
- WCSystemDistributedMapCache
- WCFlexFlowDistributedMapCache
- WCRESTTagDistributedMapCache
- WCSEOURLKeyword2URLTokenDistributedMapCache
- wCSEOURLToken2URLKeywordDistributedMapCache
- WCSEORedirectRulesDistributedMapCache
- WCSEOURLDistributedMapCache
- WCWidgetDefinitionDistributedMapCache
- WCLayoutDistributedMapCache
- WCSEOPageDefinitionDistributedMapCache
- WCPR_Cache
- DistributedMaps in the Store server
-
- WCFlexFlowDistributedMapCache
- WCStoreDistributedMapCache
- WCSEORedirectRulesDistributedMapCache
- WCSEOURLDistributedMapCache
- WCSEOURLToken2URLKeywordDistributedMapCache
- WCSEOURLKeyword2URLTokenDistributedMapCache
- WCRESTTagDistributedMapCache
- WCLayoutDistributedMapCache