Tutorial: Customizing default connectors with Ingest Profile
This tutorial demonstrates how to use Ingest profiles with default connectors to completely separate custom configurations and logic code from the product release logic. This logical separation makes it easier to upgrade to future product releases.
In the default development environment, the search profile indexes the preset attribute seller for Marketplace stores. The seller is the vendor who sells the products on the marketplace store.
In this tutorial, you learn how to index an additional attribute seller description for the marketplace store by extending the default SQL in a custom Ingest processor. You will write custom Ingest processor code in Java and build and unit test this custom Ingest processor logic exclusively in a NiFi toolkit, to extend the default connectors. Overriding the default query search profile allows you to query on the new attribute , which is indexed using the custom Ingest logic in the response fields.
Skill level
This tutorial is intended for advanced HCL Commerce developers responsible for creating and updating customizations for HCL Commerce search. The steps are mainly based on Customizing Ingest profiles.
- HCL Commerce search
- HCL Commerce services
- Ingest profiles
- Ingest system
- Java
- Maven
- NiFi
- SQL
Time required
Expect this tutorial to take 3 hours to complete. The tutorial takes longer if you explore concepts that are related to this tutorial.
Prerequisites
Perform the following steps to prepare your system for the tutorial.
- Ensure that you have an HCL Commerce development environment configured for the Elasticsearch data platform. For more information, see Setting up the HCL Commerce Developer Search environment.
- Download and extract HCL_Commerce_Search_Bundle_9.1.x.x.zip
from the HCL License and Delivery portal. After
you unzip the file, locate the file
commerce-custom-search-marketplace-seller.zip in the
extracted directory
\dependencies\nifi\Tutorials.
Extract commerce-custom-search-marketplace-seller.zip into a temporary directory in your development environment. This file is used to configure the NiFi toolkit. This file includes the sample Java code and Junit tests that you use in the tutorial.
- Enable the Marketplace preview feature in the HCL Commerce Developer Tooling environment. For more information, see Enabling Marketplace in the HCL Commerce Developer Legacy environment.
- Enable the Marketplace feature using the following Query service configuration
endpoint.
with this request body:PATCH http://QUERY_HOST:QUERY_PORT/search/resources/api/v2/configuration?nodeName=ingest&envType=auth
{ "global": { "connector": [ { "name": "attribute", "property": [ { "name": "flow.marketplace", "value": "true" } ] } ] } }
Note: All APIs mentioned throughout the tutorial must be executed by adding basic authentication for thespiuser
user, where the user isspiuser
, and the password isQxV7uCk6RRiwvPVaa4wdD78jaHi2za8ssjneNMdu3vgqi
Ensure ESite Index Model is enabled by running the following SQL query:SELECT * FROM STORECONF WHERE name = 'wc.search.CASIndexModel'
You should receive a false in the column VALUE for the storeconf table.
Trigger a full index for the EmeraldMP store (storeId=31
) using the following Ingest endpoint.POST http://INGEST_HOST:30800/connectors/auth.reindex/run?storeId=31
Once the build is complete and the NIFI console shows0/0
bytes queued data, execute the following GET commands using a REST client.- Check the response for the default search profile
HCL_V2_findProductByPartNumber_Details
, by executing the following GET call.GET http://QUERY_HOST:30920/search/resources/api/v2/products?storeId=31&seller=7000000000000007002&partNumber=MPHBA031_310301
- Check the response for the default search profile
HCL_findCatalogEntrySKUs
by executing the following GET call.GET http://QUERY_HOST:30920/search/resources/api/v2/products?storeId=31&seller=7000000000000007002&id=14503
- Check the response for the default search profile
- Prepare the development toolkit for NiFi following the
instructions in Custom NiFi processors. The NiFi
toolkit must be configured with Eclipse set to use the Java 1.8 compiler, with
Maven Version 3.8.4 or above. Set the Java build path for the imported project
to use JavaSE-1.8 libraries.Patch the Nifi container on the data platform and obtain the latest JAR files from the running patched container before you configure and build tutorial assets in the NiFi toolkit. As an example, for the Version 9.1.10 environment the JAR files are:
- commerce-search-processors-9.1.10.0.jar
- hcl-cache-core-9.1.10.0-20220322.160926-24.jar
- cf-base-9.1.10.0-20220322.160733-7.jar
- Initiate a Maven Build... on the root bundle project. In
the Project Explorer, right-click on the root
(commerce-search-custom-bundle-9.1.x.x) and select . The Edit Configuration window opens.
Enter the command clean install -U -Denforcer.skip=true into the Goals entry field and click the Run button.
Once the build is successful, the JAR files can be viewed in the target subdirectory of the
commerce-custom-search-processors
project.