Deploying Elasticsearch in a development environment
Use these instructions to deploy Elasticsearch in your .
Before you begin
-
Important: The Elasticsearch-based Search solution is resource intensive. It is recommended to dedicate standalone hardware or hosted virtual machine resources and not run it on a local development machine.
-
Before installing HCL Commerce Developer software, ensure that your HCL Commerce development environment meets the hardware and software requirements.
- For software requirements, see Supported software levels, fixes, and settings for HCL Commerce.
- For hardware requirements, see Hardware requirements for HCL Commerce.
- The requires a database.
The supports and . For more information about installing and configuring a development environment database, see Install and configure your development environment database.
- HCL Cache caches classes that can be modified in newer versions of HCL Commerce. To avoid errors in de-serializing an old version of the class, it is strongly recommended to clear Redis keys after upgrading HCL Commerce. Redis keys can be cleared with the Redis flushdb or flushall commands.
- Ensure that ample resources are allocated to Docker.
To allocate resources to Docker, right click on your Docker executable, or shortcut to it, and select Advanced. Allocate at minimum: 8 CPU cores, 32GB of memory, and 80GB of storage to ensure adequate performance.
- Ensure that the following network setting is enabled within the
Expose daemon on tcp://localhost:2375 port
settings:
- Review the list of the latest available download packages to ensure that you are obtaining the most up-to-date version of HCL Commerce software.
- Go to the .
- Search for Package ID HCL_Commerce_Devops_Version_9.1, or File ID HCL_Commerce_DevOps_9.1.x.x.bundle.
- Download the file.
- Download the Data Docker images.
- HCL_Commerce_Enterprise_9.1.x.x_Data_Query_Server_x86-64.tgz
- HCL_Commerce_Enterprise_9.1.x.x_Data_NiFi_Server_x86-64.tgz
- HCL_Commerce_Enterprise_9.1.x.x_Data_NiFi_Registry_Server_x86-64.tgz
- HCL_Commerce_Enterprise_9.1.x.x_Data_Ingest_Server_x86-64.tgz
- Optional:HCL_Commerce_Enterprise_9.1.x.x_Approval_Server_x86-64.tgz
- Optional:HCL_Commerce_Enterprise_9.1.x.x_GraphQL_Server_x86-64.tgz
docker load -i imagename
For example:
docker load -i HCL_Commerce_Enterprise_9.1.0.0_Data_Query_Server_x86-64.tgz
- Clone the project in Git from the downloaded git bundle, and check the code into your
source control management system. Run the following
command.
Where:git clone bundleName projectName
- bundleName
- The filename of the bundle you are cloning.
- projectName
- The name of the git project that you are creating.
git clone HCL_Commerce_DevOps_9.1.0.0.bundle HCL_Commerce_DevOps
Procedure
-
Deploy the Elasticsearch-based Search solution.
- Deployment option #1: Deploy the Elasticsearch-based
Search solution via the deployment script.
This option is limited to deployment within a Linux environment, due to the reliance on Bash scripts (.sh files).
- Update the env.sh bash script with the details
of your environment.
- Open env.sh for editing.
- Set the following environment details depending on your version of .
- For 9.1.12.0 and greater, the
following values must be
set:
SEARCH_ENGINE=elasticsearch DEPLOY_TYPE=toolkit LICENSE_ACCEPTED=accept TOOLKIT_HOST=<toolkitHost> TOOLKIT_HOST_IP=<toolkitHostIP> SEARCH_HOST=<searchHost> SEARCH_HOST_IP=<searchHostIP> DB_TYPE=<db2|oracle> DB_HOST=<databaseHost> DB_NAME=<databaseName> DB_USER=<databaseUser> DB_PASS=<databaseUserPassword> SEARCH_QUERY_IMAGE=<searchQueryImage> SEARCH_NIFI_IMAGE=<searchNifiImage> SEARCH_REGISTRY_IMAGE=<searchRegistryImage> SEARCH_INGEST_IMAGE=<searchIngestImage> SPIUSER_PWD_ENCRYPTED=<encryptedSpiUserPassord> ADMIN_SPIUSER_PWD=<plainTextSpiuserPassword> SPIUSER_PWD_BASE64=<base46SpiuserPassword>
- For
9.1.12.0 and greater, optionally set the following
configurations:
APPROVAL_ENABLED=<true|false> APPROVAL_APP_IMAGE=<approvalImage> SESSION_KEY_ENCRYPT=<encryptedSessionKey> GRAPHQL_ENABLED=<true|false> GRAPHQL_APP_IMAGE=<graphqlImage> ALLOW_TELEMETRY=<yes|no> CACHE_ENABLED=<true|false> CACHE_APP_IMAGE=<cacheManagerImage>
- For 9.1.0.0 to
9.1.11.0:
SEARCH_ENGINE=elasticsearch DEPLOY_TYPE=data DATA_FOR_TOOLKIT=true LICENSE_ACCEPTED=accept TOOLKIT_HOST=<toolkitHost> DB_TYPE=<db2|oracle> DB_HOST=<databaseHost> DB_NAME=<databaseName> DB_USER=<databaseUser> DB_PASS=<databaseUserPassword> SEARCH_QUERY_IMAGE=<searchQueryImage> SEARCH_NIFI_IMAGE=<searchNifiImage> SEARCH_REGISTRY_IMAGE=<searchRegistryImage> SEARCH_INGEST_IMAGE=<searchIngestImage> SPIUSER_PWD_ENCRYPTED=<encryptedSpiUserPassord> ADMIN_SPIUSER_PWD=<plainTextSpiuserPassword> SPIUSER_PWD_BASE64=<base46SpiuserPassword>
Note:- For more information on the env.conf configuration file, review The Docker Compose deployment env.sh configuration file.
- For 9.1.12.0 and greater, the
following values must be
set:
- Save and close the file.
- Back up the env.sh script containing your environment details for future deployment.
- If you are using an database, ensure that there is a copy of the required JDBC driver (ojdbc8.jar) located within the volumes/jdbc/ directory.
- Run the deployment bash
script.
./deploy.sh
This script produces an Elasticsearch-based Search Docker Compose file that is based on the environment configuration in env.sh.- For 9.1.12.0 and greater, this file is docker-compose-toolkit.yml.
- For 9.1.0.0 to 9.1.11.0, this file is docker-compose-data.yml.
The deployment will set the project to
commerce
, so the created Docker services will havecommerce_
prefix. For example,commerce_nifi_1
.The deployment script will automatically start the deployment.
- Update the env.sh bash script with the details
of your environment.
- Deployment option #2: Deploy the platform
manually.
This option is the only deployment that is compatible with a Windows host operating system, due to its non-reliance on Bash scripts (.sh files).
- Create a Docker Compose deployment file for your environment.
- Select a deployment Docker Compose template file based on the database that
you will use for the deployment.
The template files are follow the naming convention: docker-compose-elasticsearch-data-<DB_TYPE>-toolkit-template.yml.
Where <DB_TYPE> can be db2 or oracle.
- Copy the template into your Docker Compose deployment file (docker-compose-toolkit.yml).
- Add optional services to your Docker Compose deployment file
(docker-compose-toolkit.yml).
- GraphQL (docker-compose-elasticsearch-svc-graphql-toolkit-template.yml)
- Cache Manager (docker-compose-elasticsearch-data-toolkit-cache-app-svc-template.yaml)
- (
docker-compose-svc-approval-toolkit-template.yml)Note: In addition to including the content from this Docker Compose template, to enable the you must overwrite volumes/approval-app/boot-config/app-secure.properties with the contents of volumes/approval-app/boot-config/app-secure.properties.nonssl.
- Select a deployment Docker Compose template file based on the database that
you will use for the deployment.
- Update the env.sh bash script with the details
of your environment.
- Open env.sh for editing.
- Set the following environment details depending on your version of .
- For 9.1.12.0 and greater, the
following values must be
set:
SEARCH_ENGINE=elasticsearch DEPLOY_TYPE=toolkit LICENSE_ACCEPTED=accept TOOLKIT_HOST=<toolkitHost> TOOLKIT_HOST_IP=<toolkitHostIP> SEARCH_HOST=<searchHost> SEARCH_HOST_IP=<searchHostIP> DB_TYPE=<db2|oracle> DB_HOST=<databaseHost> DB_NAME=<databaseName> DB_USER=<databaseUser> DB_PASS=<databaseUserPassword> SEARCH_QUERY_IMAGE=<searchQueryImage> SEARCH_NIFI_IMAGE=<searchNifiImage> SEARCH_REGISTRY_IMAGE=<searchRegistryImage> SEARCH_INGEST_IMAGE=<searchIngestImage> SPIUSER_PWD_ENCRYPTED=<encryptedSpiUserPassord> ADMIN_SPIUSER_PWD=<plainTextSpiuserPassword> SPIUSER_PWD_BASE64=<base46SpiuserPassword>
- For
9.1.12.0 and greater, optionally set the following
configurations:
APPROVAL_ENABLED=<true|false> APPROVAL_APP_IMAGE=<approvalImage> SESSION_KEY_ENCRYPT=<encryptedSessionKey> GRAPHQL_ENABLED=<true|false> GRAPHQL_APP_IMAGE=<graphqlImage> ALLOW_TELEMETRY=<yes|no> CACHE_ENABLED=<true|false> CACHE_APP_IMAGE=<cacheManagerImage>
- For 9.1.0.0 to
9.1.11.0:
SEARCH_ENGINE=elasticsearch DEPLOY_TYPE=data DATA_FOR_TOOLKIT=true LICENSE_ACCEPTED=accept TOOLKIT_HOST=<toolkitHost> DB_TYPE=<db2|oracle> DB_HOST=<databaseHost> DB_NAME=<databaseName> DB_USER=<databaseUser> DB_PASS=<databaseUserPassword> SEARCH_QUERY_IMAGE=<searchQueryImage> SEARCH_NIFI_IMAGE=<searchNifiImage> SEARCH_REGISTRY_IMAGE=<searchRegistryImage> SEARCH_INGEST_IMAGE=<searchIngestImage> SPIUSER_PWD_ENCRYPTED=<encryptedSpiUserPassord> ADMIN_SPIUSER_PWD=<plainTextSpiuserPassword> SPIUSER_PWD_BASE64=<base46SpiuserPassword>
Note:- For more information on the env.conf configuration file, review The Docker Compose deployment env.sh configuration file.
- For 9.1.12.0 and greater, the
following values must be
set:
- Save and close the file.
- Back up the env.sh script containing your environment details for future deployment.
- Copy your configured env.sh values into the Docker .env file on the same level as your Docker Compose file (docker-compose-toolkit.yml). This allows for docker-compose to use these environment variables in the deployment.
- Set file permission for your persistent volumes.
Some volumes are mounted to your running containers and require write permissions.
Ensure that you recursively set write access to /volumes/, to ensure that it and all of the contained sub-directories are writable.
- Run the following command to start the Elasticsearch Search
solution.
docker-compose up -d
- Create a Docker Compose deployment file for your environment.
- Legacy deployment option: Deploy a legacy version of the platform manually.
In the event that you need to support an older version of 9.1, this option is the only method available for deploying its corresponding Elasticsearch-based solution.
- This deployment option is no longer supported with 9.1.6.0 and greater.
- This is the only deployment option for versions 9.1.0.0 through 9.1.5.0.
- From the docker-compose folder, copy the file docker-compose-elasticSearch-data-db2-toolkit-template.yml or docker-compose-elasticSearch-data-oracle-toolkit-template.yml to docker-compose.yml.
- Update the .yml image tag for all HCL Commerce Docker
containers.
- Determine the image repository and tag by running the Docker images.
- Update each image tag in the .yml file with the its
respective repository:tag. For example,
image: commerce/search-nifi-app:9.1.0.0
- Accept the LICENSE parameter for all
docker containers, for example:
- LICENSE=accept
- Update the following configuration for Nifi to point to the
database.
AUTH_JDBC_URL: "jdbc:DBType://DBHost:50000/mall" AUTH_JDBC_DRIVER_LOCATION: "/opt/nifi/nifi-current/lib/DBDriver" AUTH_JDBC_USER_NAME: "DBUser" AUTH_JDBC_USER_PASSWORD: "DBUserPassword" NLP_ENABLE_LANGUAGE_CODE: "nlpEnableLanguageCode"
For example:
AUTH_JDBC_URL: "jdbc:db2://tk.hclcommerce.com:50000/mall" AUTH_JDBC_DRIVER_LOCATION: "/opt/nifi/nifi-current/lib/db2jcc4.jar" AUTH_JDBC_USER_NAME: "wcs" AUTH_JDBC_USER_PASSWORD: "wcs1" NLP_ENABLE_LANGUAGE_CODE: "EN"
AUTH_JDBC_URL: "jdbc:oracle:thin:@//tk.hclcommerce.com:1521/orclpdb1" AUTH_JDBC_DRIVER_CLASSNAME: "oracle.jdbc.driver.OracleDriver" AUTH_JDBC_DRIVER_LOCATION: "/opt/nifi/nifi-current/lib/ojdbc8.jar" AUTH_JDBC_USER_NAME: "wcs" AUTH_JDBC_USER_PASSWORD: "wcs1" NLP_ENABLE_LANGUAGE_CODE: "EN"
- Update the configuration for
data-query.
TX_HOST: "TXHost"
For example:
TX_HOST: "tk.hclcommerce.com"
- Create the following sub-directories within in the same directory where
docker-compose.yml is located, and ensure that their
permissions are set to
777
.Important: If you are upgrading from 9.1.0.0 or 9.1.1.0 to a newer version of , or if you have any issues related to the search index or NiFi service connector, you must delete the contents of these directories.- nifiLogs
- elasticsearch/data
- elasticsearch/config
- zookeeper
- nifi/data
- redis/data
- Copy the hcl-cache-search folder to the location where the Docker Compose file (docker-compose.yml) is located.
- If you are using an or database, ensure that there is a copy of the required JDBC driver (ojdbc8.jar) in the same directory where the Docker compose file (docker-compose.yml) is located.
- Run the following command to start the Elasticsearch Search
solution.
docker-compose up -d
- Deployment option #1: Deploy the Elasticsearch-based
Search solution via the deployment script.
- Verify that all the containers are up and healthy.