HCL Commerce Version 9.1.16.0 or later

Enabling Solr search sharding within a Kubernetes deployment

Solr search sharding is included as an optional configuration in the HCL Commerce 9.1.16.0 Helm Chart. This enables Solr-based search deployments with large indexes to enable multiple Java Virtual Machines (JVMs) to complete indexing work in parallel, reducing indexing time, and alleviating any resource issues that can be encountered when using a single JVM.

By default, the Helm Chart supports three shards, shardA, shardB and shardC. If additional shards are required, see Adding additional shards to add them manually.

To learn more about Solr search sharding, see Solr search sharding.

Procedure

  1. Modify your HCL Commerce Kubernetes deployment configuration to enable parallel pre-processing and distributed indexing for Solr search.
    1. Enable shardA, shardB and shardC under searchAppMaster in your deployment Values.yaml file.

      With this configuration, multiple search servers are deployed with the same configurations as searchAppMaster (with the exception of persistence, nodeLabel, nodeSelector and coresSharingPersistentVolumeClaim).

    2. Create ReadWriteMany persistent volumes for searchAppMaster, shardA, shardB and shardC.

      See Setting up persistent storage volumes for a Kubernetes deployment.

  2. Start your HCL Commerce deployment.
  3. Modify your ts-utils Solr-based search index pre-processing configuration files.
    Tip: If you want to persist the changes made to your properties files within your Utility server Docker container container, you must build a customized ts-utils image and use this customized image for your deployment.
    1. Enter the running Utility server Docker container.

      See Entering Docker containers.

    2. Open the following configuration files for modification:
      • /opt/WebSphere/CommerceServer90/properties/parallelprocess/di-parallel-process-multijvms.properties
      • /opt/WebSphere/CommerceServer90/properties/parallelprocess/password.properties
    3. Within di-parallel-process-multijvms.properties:
      1. Configure the hostname and port for the individual shard servers as follows.
        Shard.A.common.index-server-name=shard_a
Shard.A.common.index-server-port=3738
…
Shard.B.common.index-server-name=shard_b
Shard.B.common.index-server-port=3738
        Each shard server name should be the same as the hostname/alias configured within your deployment.
        Note: Server names within this file are the same as service names.
        For example:
        Global.index-server-name=ivtsolrdb2authsearch-app-master.solrdb2.svc.cluster.local
Shard.A.common.index-server-name=ivtsolrdb2authsearch-app-shard-a.solrdb2.svc.cluster.local
        Where:
        • The tenant name is ivt.
        • The environment name is solrdb2
        • The environment type is auth.
        • The namespace is solrdb2.
      2. Configure the shard index core directory as follows.
        Shard.A.en_US.unstructured-index-core-dir=/shard_a/index/solr/MC_10001/en_US/Unstructured_A/
Shard.A.en_US.structured-index-core-dir=/shard_a/index/solr/MC_10001/en_US/CatalogEntry_A/
…
Shard.B.en_US.unstructured-index-core-dir=/shard_b/index/solr/MC_10001/en_US/Unstructured_B/
Shard.B.en_US.structured-index-core-dir=/shard_b/index/solr/MC_10001/en_US/CatalogEntry_B/
        Note: The directory should be the absolute path inside each shard container.

        You can automate the shard configuration process. This is useful if, for example, you expect to create a large number of shards. The auto-sharding process will automatically configure properties such as preprocessing-start-range-value, preprocessing-end-range-value, index-core-name and index-core-dir.

        For information on how to set up auto-sharding, see Sharding input properties file.

    4. Within password.properties:

      Ensure that the values for dbUserPassword and searchAdminPassword are set correctly.

    5. Save and close the files.
  4. Start the indexing process with sharding in multiple JVMs enabled.
    1. Navigate to the /opt/WebSphere/CommerceServer90/bin/ directory.
    2. Run the following command.
      ./di-parallel-process.sh /opt/WebSphere/CommerceServer90/properties/parallelprocess/di-parallel-process-multijvms.properties
      For more information, see Running utilities from the Utility server Docker container.

Results

Your HCL Commerce deployment contains your newly configured Solr search shards.