Deploying Elasticsearch in a development environment

Use these instructions to deploy Elasticsearch in your .

Before you begin

Important: The Elasticsearch-based Search solution is resource intensive. It is recommended to dedicate standalone hardware or hosted virtual machine resources and not run it on a local development machine.
Before installing HCL Commerce Developer software, ensure that your HCL Commerce development environment meets the hardware and software requirements.
- For software requirements, see Supported software levels, fixes, and settings for HCL Commerce.
- For hardware requirements, see Hardware requirements for HCL Commerce.
The requires a database.
The supports and . For more information about installing and configuring a development environment database, see Install and configure your development environment database.
HCL Cache caches classes that can be modified in newer versions of HCL Commerce. To avoid errors in de-serializing an old version of the class, it is strongly recommended to clear Redis keys after upgrading HCL Commerce. Redis keys can be cleared with the Redis flushdb or flushall commands.

Note: When using Microsoft Windows:

Ensure that ample resources are allocated to Docker.
To allocate resources to Docker, right click on your Docker executable, or shortcut to it, and select Advanced. Allocate at minimum: 8 CPU cores, 32GB of memory, and 80GB of storage to ensure adequate performance.
Ensure that the following network setting is enabled within the Docker settings > General settings:
```
Expose daemon on tcp://localhost:2375 port
```

To start the deployment process, download the required software.

Review the list of the latest available download packages to ensure that you are obtaining the most up-to-date version of HCL Commerce software.
Go to the .
Search for Package ID HCL_Commerce_Devops_Version_9.1, or File ID HCL_Commerce_DevOps_9.1.x.x.bundle.
Download the file.
Download the Data Docker images.
- HCL_Commerce_Enterprise_9.1.x.x_Data_Query_Server_x86-64.tgz
- HCL_Commerce_Enterprise_9.1.x.x_Data_NiFi_Server_x86-64.tgz
- HCL_Commerce_Enterprise_9.1.x.x_Data_NiFi_Registry_Server_x86-64.tgz
- HCL_Commerce_Enterprise_9.1.x.x_Data_Ingest_Server_x86-64.tgz
- Optional:HCL_Commerce_Enterprise_9.1.x.x_Approval_Server_x86-64.tgz
- Optional:HCL_Commerce_Enterprise_9.1.x.x_GraphQL_Server_x86-64.tgz
Load each Docker image. For each downloaded image file, run the following command.
```
docker load -i imagename
```
For example:
```
docker load -i HCL_Commerce_Enterprise_9.1.0.0_Data_Query_Server_x86-64.tgz
```
Clone the project in Git from the downloaded git bundle, and check the code into your source control management system. Run the following command.
```
git clone bundleName projectName
```
Where:

bundleName

The filename of the bundle you are cloning.

projectName

The name of the git project that you are creating.

For example:
```
git clone HCL_Commerce_DevOps_9.1.0.0.bundle HCL_Commerce_DevOps
```

Procedure

Deploy the Elasticsearch-based Search solution.
- Deployment option #1: Deploy the Elasticsearch-based Search solution via the deployment script.
  This option is limited to deployment within a Linux environment, due to the reliance on Bash scripts (.sh files).
  1. Update the env.sh bash script with the details of your environment.
    1. Open env.sh for editing.
    2. Set the following environment details depending on your version of .
      - For 9.1.12.0 and greater, the following values must be set:
        SEARCH_ENGINE=elasticsearch DEPLOY_TYPE=toolkit LICENSE_ACCEPTED=accept TOOLKIT_HOST=<toolkitHost> TOOLKIT_HOST_IP=<toolkitHostIP> SEARCH_HOST=<searchHost> SEARCH_HOST_IP=<searchHostIP> DB_TYPE=<db2|oracle> DB_HOST=<databaseHost> DB_NAME=<databaseName> DB_USER=<databaseUser> DB_PASS=<databaseUserPassword> SEARCH_QUERY_IMAGE=<searchQueryImage> SEARCH_NIFI_IMAGE=<searchNifiImage> SEARCH_REGISTRY_IMAGE=<searchRegistryImage> SEARCH_INGEST_IMAGE=<searchIngestImage> SPIUSER_PWD_ENCRYPTED=<encryptedSpiUserPassord> ADMIN_SPIUSER_PWD=<plainTextSpiuserPassword> SPIUSER_PWD_BASE64=<base46SpiuserPassword>
      - For 9.1.12.0 and greater, optionally set the following configurations:
        APPROVAL_ENABLED=<true|false> APPROVAL_APP_IMAGE=<approvalImage> SESSION_KEY_ENCRYPT=<encryptedSessionKey> GRAPHQL_ENABLED=<true|false> GRAPHQL_APP_IMAGE=<graphqlImage> ALLOW_TELEMETRY=<yes|no> CACHE_ENABLED=<true|false> CACHE_APP_IMAGE=<cacheManagerImage>
      - For 9.1.0.0 to 9.1.11.0:
        SEARCH_ENGINE=elasticsearch DEPLOY_TYPE=data DATA_FOR_TOOLKIT=true LICENSE_ACCEPTED=accept TOOLKIT_HOST=<toolkitHost> DB_TYPE=<db2|oracle> DB_HOST=<databaseHost> DB_NAME=<databaseName> DB_USER=<databaseUser> DB_PASS=<databaseUserPassword> SEARCH_QUERY_IMAGE=<searchQueryImage> SEARCH_NIFI_IMAGE=<searchNifiImage> SEARCH_REGISTRY_IMAGE=<searchRegistryImage> SEARCH_INGEST_IMAGE=<searchIngestImage> SPIUSER_PWD_ENCRYPTED=<encryptedSpiUserPassord> ADMIN_SPIUSER_PWD=<plainTextSpiuserPassword> SPIUSER_PWD_BASE64=<base46SpiuserPassword>
      Note:
      
      For more information on the env.conf configuration file, review The Docker Compose deployment env.sh configuration file.
    3. Save and close the file.
    4. Back up the env.sh script containing your environment details for future deployment.
  2. If you are using an database, ensure that there is a copy of the required JDBC driver (ojdbc8.jar) located within the volumes/jdbc/ directory.
  3. Run the deployment bash script.
```
./deploy.sh
```
    This script produces an Elasticsearch-based Search Docker Compose file that is based on the environment configuration in env.sh.
    - For 9.1.12.0 and greater, this file is docker-compose-toolkit.yml.
    - For 9.1.0.0 to 9.1.11.0, this file is docker-compose-data.yml.
    The deployment will set the project to commerce, so the created Docker services will have commerce_ prefix. For example, commerce_nifi_1.
    The deployment script will automatically start the deployment.
- Deployment option #2: Deploy the platform manually.
  This option is the only deployment that is compatible with a Windows host operating system, due to its non-reliance on Bash scripts (.sh files).
  1. Create a Docker Compose deployment file for your environment.
    1. Select a deployment Docker Compose template file based on the database that you will use for the deployment.
      The template files are follow the naming convention: docker-compose-elasticsearch-data-<DB_TYPE>-toolkit-template.yml.
      Where <DB_TYPE> can be db2 or oracle.
    2. Copy the template into your Docker Compose deployment file (docker-compose-toolkit.yml).
    3. Add optional services to your Docker Compose deployment file (docker-compose-toolkit.yml).
      - GraphQL (docker-compose-elasticsearch-svc-graphql-toolkit-template.yml)
      - Cache Manager (docker-compose-elasticsearch-data-toolkit-cache-app-svc-template.yaml)
    4. ( docker-compose-svc-approval-toolkit-template.yml)
      Note: In addition to including the content from this Docker Compose template, to enable the you must overwrite volumes/approval-app/boot-config/app-secure.properties with the contents of volumes/approval-app/boot-config/app-secure.properties.nonssl.
  2. Update the env.sh bash script with the details of your environment.
    1. Open env.sh for editing.
    2. Set the following environment details depending on your version of .
      - For 9.1.12.0 and greater, the following values must be set:
        SEARCH_ENGINE=elasticsearch DEPLOY_TYPE=toolkit LICENSE_ACCEPTED=accept TOOLKIT_HOST=<toolkitHost> TOOLKIT_HOST_IP=<toolkitHostIP> SEARCH_HOST=<searchHost> SEARCH_HOST_IP=<searchHostIP> DB_TYPE=<db2|oracle> DB_HOST=<databaseHost> DB_NAME=<databaseName> DB_USER=<databaseUser> DB_PASS=<databaseUserPassword> SEARCH_QUERY_IMAGE=<searchQueryImage> SEARCH_NIFI_IMAGE=<searchNifiImage> SEARCH_REGISTRY_IMAGE=<searchRegistryImage> SEARCH_INGEST_IMAGE=<searchIngestImage> SPIUSER_PWD_ENCRYPTED=<encryptedSpiUserPassord> ADMIN_SPIUSER_PWD=<plainTextSpiuserPassword> SPIUSER_PWD_BASE64=<base46SpiuserPassword>
      - For 9.1.12.0 and greater, optionally set the following configurations:
        APPROVAL_ENABLED=<true|false> APPROVAL_APP_IMAGE=<approvalImage> SESSION_KEY_ENCRYPT=<encryptedSessionKey> GRAPHQL_ENABLED=<true|false> GRAPHQL_APP_IMAGE=<graphqlImage> ALLOW_TELEMETRY=<yes|no> CACHE_ENABLED=<true|false> CACHE_APP_IMAGE=<cacheManagerImage>
      - For 9.1.0.0 to 9.1.11.0:
        SEARCH_ENGINE=elasticsearch DEPLOY_TYPE=data DATA_FOR_TOOLKIT=true LICENSE_ACCEPTED=accept TOOLKIT_HOST=<toolkitHost> DB_TYPE=<db2|oracle> DB_HOST=<databaseHost> DB_NAME=<databaseName> DB_USER=<databaseUser> DB_PASS=<databaseUserPassword> SEARCH_QUERY_IMAGE=<searchQueryImage> SEARCH_NIFI_IMAGE=<searchNifiImage> SEARCH_REGISTRY_IMAGE=<searchRegistryImage> SEARCH_INGEST_IMAGE=<searchIngestImage> SPIUSER_PWD_ENCRYPTED=<encryptedSpiUserPassord> ADMIN_SPIUSER_PWD=<plainTextSpiuserPassword> SPIUSER_PWD_BASE64=<base46SpiuserPassword>
      Note:
      
      For more information on the env.conf configuration file, review The Docker Compose deployment env.sh configuration file.
    3. Save and close the file.
    4. Back up the env.sh script containing your environment details for future deployment.
  3. Copy your configured env.sh values into the Docker .env file on the same level as your Docker Compose file (docker-compose-toolkit.yml). This allows for docker-compose to use these environment variables in the deployment.
  4. Set file permission for your persistent volumes.
    Some volumes are mounted to your running containers and require write permissions.
    Ensure that you recursively set write access to /volumes/, to ensure that it and all of the contained sub-directories are writable.
  5. Run the following command to start the Elasticsearch Search solution.
```
docker-compose up -d
```
- Legacy deployment option: Deploy a legacy version of the platform manually.
  In the event that you need to support an older version of 9.1, this option is the only method available for deploying its corresponding Elasticsearch-based solution.
  - This deployment option is no longer supported with 9.1.6.0 and greater.
  - This is the only deployment option for versions 9.1.0.0 through 9.1.5.0.
  1. From the docker-compose folder, copy the file docker-compose-elasticSearch-data-db2-toolkit-template.yml or docker-compose-elasticSearch-data-oracle-toolkit-template.yml to docker-compose.yml.
  2. Update the .yml image tag for all HCL Commerce Docker containers.
    1. Determine the image repository and tag by running the Docker images.
    2. Update each image tag in the .yml file with the its respective repository:tag. For example, image: commerce/search-nifi-app:9.1.0.0
  3. Accept the LICENSE parameter for all docker containers, for example: - LICENSE=accept
  4. Update the following configuration for Nifi to point to the database.
```
  AUTH_JDBC_URL: "jdbc:DBType://DBHost:50000/mall"
  AUTH_JDBC_DRIVER_LOCATION: "/opt/nifi/nifi-current/lib/DBDriver"
  AUTH_JDBC_USER_NAME: "DBUser"
  AUTH_JDBC_USER_PASSWORD: "DBUserPassword"
  NLP_ENABLE_LANGUAGE_CODE: "nlpEnableLanguageCode"
```
    For example:
```
AUTH_JDBC_URL: "jdbc:db2://tk.hclcommerce.com:50000/mall"
  AUTH_JDBC_DRIVER_LOCATION: "/opt/nifi/nifi-current/lib/db2jcc4.jar"
  AUTH_JDBC_USER_NAME: "wcs"
  AUTH_JDBC_USER_PASSWORD: "wcs1"
  NLP_ENABLE_LANGUAGE_CODE: "EN"
```
```
AUTH_JDBC_URL: "jdbc:oracle:thin:@//tk.hclcommerce.com:1521/orclpdb1"
AUTH_JDBC_DRIVER_CLASSNAME: "oracle.jdbc.driver.OracleDriver"
AUTH_JDBC_DRIVER_LOCATION: "/opt/nifi/nifi-current/lib/ojdbc8.jar"
AUTH_JDBC_USER_NAME: "wcs"
AUTH_JDBC_USER_PASSWORD: "wcs1"
NLP_ENABLE_LANGUAGE_CODE: "EN"
```
  5. Update the configuration for data-query.
```
  TX_HOST: "TXHost"
```
    For example:
```
TX_HOST: "tk.hclcommerce.com"
```
  6. Create the following sub-directories within in the same directory where docker-compose.yml is located, and ensure that their permissions are set to 777.
    Important: If you are upgrading from 9.1.0.0 or 9.1.1.0 to a newer version of , or if you have any issues related to the search index or NiFi service connector, you must delete the contents of these directories.
    - nifiLogs
    - elasticsearch/data
    - elasticsearch/config
    - zookeeper
    - nifi/data
    - redis/data
  7. Copy the hcl-cache-search folder to the location where the Docker Compose file (docker-compose.yml) is located.
  8. If you are using an or database, ensure that there is a copy of the required JDBC driver (ojdbc8.jar) in the same directory where the Docker compose file (docker-compose.yml) is located.
  9. Run the following command to start the Elasticsearch Search solution.
```
docker-compose up -d
```
Verify that all the containers are up and healthy.

What to do next

Follow the steps in Building the Elasticsearch index to complete your environment setup.