Deploy custom Ingest logic on ElasticSearch server for Commerce Development environment
In this topic, you would extend the default connector by deploying the custom ingest logic in NiFi container on the data platform and then creating a custom Ingest profile to bind it to the default connector pipe.
Procedure
- Copy
commerce-custom-search-marketplace-seller-9.1.10.0.jar
from the target directory under
commerce-custom-search-marketplace-seller
project in NiFi toolkit, and ingest-api.jar
from commerce-custom-search-marketplace-seller
project’s libs directory, to the nifi
container’s libs directory on the Elasticsearch
data server.
docker cp "/root/commerce-custom-search-marketplace-seller-9.1.10.0.jar" NIFI_CONTAINER_ID:/opt/nifi/nifi-current/lib docker cp "/root/ingest-api.jar" NIFI_CONTAINER_ID:/opt/nifi/nifi-current/lib
Note: From V9.1.12.0 onwards, ingest-api.jar is not made available inside tutorial assets, and hence following docker copy command need not be executed for copying it over to the NiFi container's lib directory.docker cp "/root/ingest-api.jar" NIFI_CONTAINER_ID:/opt/nifi/nifi-current/lib
Note: (Optional) This customization tutorial is independent of the commerce-custom-search-processors-nar project. However, if your customization depends on the commerce-custom-search-processors-nar project, transfer the NAR file from the commerce-custom-search-processors-nar project’s target directory to the designated extension directory on the Elasticsearch server and mount this file as a volume in the NiFi Docker container.Ignore the NAR files generated in projects other than commerce-custom-search-processors-nar. For more information, see Building and deploying a custom NAR file.
or example, you could use the following command to mount the NAR file that has been added to the/extensions folder, instead of mounting the volume folder:docker run -it -p 30600:30600 --name elasticsearch_nifi_1 -v /{docker-compose dir}/volumes/nifi-ext/commerce-custom-search-processors-nar-9.1.x.x:/opt/nifi/nifi-current/extensions/commerce-custom-search-processors-nar-9.1.x.x search-nifi-app:v9-latest
Where elasticsearch_nifi_1 is the NiFi container from Elasticsearch server and search-nifi-app:v9-latest is the NiFi image.
- Restart the Nifi container.
In this topic, you have extended the default connector by deploying custom ingest logic in the NiFi container. Additionally, you have created a custom Ingest profile to bind it to the default connector pipe.