Deploying Ingest Profile custom extensions on production runtime environment
This topic explains how to deploy a custom search NIFI image on production.
Procedure
- Create a custom
search-nifi-app
Docker image.For runtime deployment of Ingest customization extensions, create a custom Docker image forsearch-nifi-app
to include the customizations created using the NiFi toolkit.- Create a Dockerfile Docker configuration file with the
following content.
FROM search-nifi-app docker image COPY CusDeploy/ /opt/nifi/nifi-current/lib
Where CusDeploy/ contains the commerce-custom-search-marketplace-seller-9.1.10.0.jar that you built in Testing custom ingest logic in NiFi toolkit and building deployable JAR.
If patch needs to be applied:FROM comlnx94.prod.hclpnp.com/9.1.10.0/search-nifi-app:v9-20220322-1524 COPY patches / /opt/nifi/nifi-current/patches WORKDIR /opt/nifi/scripts RUN ./patch.sh COPY CusDeploy/ /opt/nifi/nifi-current/lib
Where patches/ directory contains the patch (jar) to be applied in V9.1.10.0 and CusDeploy/ to contain commerce-custom-search-marketplace-seller-9.1.10.0.jar built in Testing custom ingest logic in NiFi toolkit and building deployable JAR.
FROM comlnx94.prod.hclpnp.com/9.1.10.0/search-nifi-app:v9-20220322-1524 COPY CusDeploy/ /opt/nifi/nifi-current/lib
Where CusDeploy/ to contain commerce-custom-search-marketplace-seller-9.1.10.0.jar built in Testing custom ingest logic in NiFi toolkit and building deployable JAR.
- Prepare the custom deployment files.
- Create the CusDeploy subdirectory in the same directory as the Dockerfile created in Create a custom NiFi processor by extending the default Ingest processor in the NiFi Toolkit.
- Copy the commerce-custom-search-marketplace-seller-9.1.x.x.jar into the CusDeploy directory.
- Copy the
ingest-api.jar obtained from
the libs directory under the
extracted
commerce-custom-search-marketplace-seller.zip
project template into the
CusDeploy directory.Note: From V9.1.12.0 onwards, ingest-api.jar is not made available inside tutorial assets.
- Build the custom NiFi server
(search-nifi-app) Docker image.
- Open a command prompt and navigate to the directory containing your Dockerfile, and its CusDeploy subdirectory.
- Run the Docker build command to build a custom
Docker image for the NiFi server
(search-nifi-app)
For example:docker build -t search-nifi-app:tag
docker build -t search-nifi-app:nifi-extension .
Note:- Ensure that the period is present at the end of your Docker build command.
- The tag, like the deployment package build label, is used to identify the custom Docker image. You can modify this at any time to help identify the generated Docker image.
- Create a Dockerfile Docker configuration file with the
following content.
- Deploy the runtime Elasticsearch data Server through CI/CD using custom
Docker image created in Create
a custom NiFi processor by extending the default Ingest processor in
the NiFi Toolkit.This task covers the minimum requirements for a deployment that is based on Docker Compose.
- Set up an HCL Commerce production environment using the Docker Compose deployment method. For information on setting up this environment, see Deploying HCL Commerce Version 9.1 with Docker Compose (for non-production usage).
- (Optional) For your custom NiFi server Docker image (search-nifi-app), ensure that you have the following deployment parameters correctly defined under the environment section of your docker-compose configuration file.
If your customization depends on the commerce-custom-search-processors-nar project, transfer the NAR file from the commerce-custom-search-processors-nar project’s "target" directory to the designated extension directory on Elasticsearch server and mount this file as a volume in the NiFi Docker container. For detailed instructions, refer to topic Building and deploying a custom NAR file.volumes: - ./volumes/hcl-cache-local-redis/cache_cfg-ext.yaml:/SETUP/hcl-cache/cache_cfg-ext.yaml - ./volumes/hcl-cache-local-redis/redis_cfg.yaml:/SETUP/hcl-cache/redis_cfg.yaml - ./volumes/nifi-ext/commerce-custom-search-processors-nar-9.1.10.0.nar:/opt/nifi/nifi-current/extensions/commerce-custom-search-processors-nar-9.1.10.0.nar
following steps are to be performed on the runtime environment the same way that they were performed on the Development environment:- Create an Ingest profile and bind it to a default connector (Creating a custom Ingest profile and binding to the default Connector Pipe).
- Override the default search query profile to include the additional seller_description.raw attribute in the query response. (Overriding Query search profiles to modify the response fields).
- Validate that the search profile response contains additional the attribute seller_description.raw (Testing the Ingest profile extensions).
This topic covered how to create a custom search NiFi image, including the ingest logic, and deploy it to production. The next step is to create a custom ingest profile and bind it to a default connector.