Creating a custom Ingest profile and binding to the default connector pipe

In this topic, you will learn to create a custom Ingest profile and bind it to the default connector pipe.

Procedure

To create a custom Ingest profile and bind it to the default connector pipe follow the steps below.

Using a REST client, issue the following POST command to create a DatabaseProductStage1a ingest profile.
```
POST http://QUERY_HOST:QUERY_PORT/search/resources/api/v2/documents/profiles/DatabaseProductStage1a?profileType=Ingest
```
Below is the request body. Please note it obtains the fully qualified class names of the custom provider and post processor created earlier in the commerce-custom-search-marketplace-seller-9.1.x.x project.
```
{
   "profileName": "DatabaseProductStage1a",
   "provider": [
         "com.mycompany.data.ingest.product.providers.ChangeSQL"
   ],
   "preprocessor": [],
   "postprocessor": [
  "com.mycompany.data.ingest.product.processors.ChangeDocument"
   ]
}	
```
Following is the expected response:
```
{
  "Profile created with name": "DatabaseProductStage1a"
}
```
Restart the query-app container.
Bind the custom Ingest profile to the connector pipe. Use the following steps. For instructions on how to use an Ingest endpoint to update the Ingest profile binding as a variable to a connector pipe, see Updating NiFi process group, processor, controller service using Ingest connector descriptor.
- Open your Nifi user interface and navigate to NiFi Flow > auth.reindex - Product Stage 1a (Main Document) > Create Product Document > SCROLL SQL.
- Right click on the empty canvas and click on Variables.
- Create a new variable with the name ingest.profile.name and a value ofDatabaseProductStage1a. Click on Apply button.
- Restart the Nifi container.
- Once the NiFi user interface is running and there are no errors or invalid components, trigger a full index for storeId=31. This can be done using the following ingest endpoint.
```
POST http://INGEST_HOST:INGEST_PORT/connectors/auth.reindex/run?storeId=31
```
- Check completion of the index using the runid obtained in the response of the build index, as in the following example:
```
GET http://INGEST_HOST:INGEST_PORT/connectors/auth.reindex/runs/runid/status
```
  The seller_description.raw attribute is also indexed by the build index and can also be validated from the NiFi console.
  Navigate to NiFi Flow > auth.reindex - Product Stage 1a (Main Document) > Create Product Document > Create Product Document from Database. Right-click on Transform Document > Create Product from Database > View Data Provenance.
- Click on View Details icon for CONTENT_MODIFIED under the Type column.
- Select the Content tab in the Provenance Event dialog box and click on the View button.

You learned how to create a custom Ingest profile and bind it to the default connector pipe in this topic.

Overriding Query search profiles to modify the response fields

In this topic, you will learn how to override query search profiles to modify the response fields.

Procedure

Override query search profiles to modify the response fields.

You can override default search profiles by creating a custom profile using the same name as the default profile. If a custom profile with the same name as a default profile is available, the custom profile is used and the default profile is ignored. For detailed steps on overriding search profiles, see Setting up your custom search profile.

Override the default search query profile to include the additional seller_description.raw attribute in the query response.
Extend the default HCL_findCatalogEntrySKUs search profile by adding seller_description.raw into the responseFields section of the existing search profile.
```
GET http://QUERY_HOST:30920/search/resources/api/v2/documents/profiles/HCL_findCatalogEntrySKUs
```

Execute the GET call to retrieve the default query profile, and execute the PUT call to create the custom query profile using the response of the GET call as a request Body after adding the seller.description.raw attribute in JSON. This would extend the default query profile as follows:

PUT http://QUERY_HOST:30920/search/resources/api/v2/documents/profiles/HCL_findCatalogEntrySKUs


{
"profileName" : "HCL_findCatalogEntrySKUs",

"indexName" : "product",
	
"query" : {
		
"params" : [
{"maxRows" : "1200"}
],
			
"queryFields" : ["relationship.product.id"],
		
"provider" : [
"com.hcl.commerce.search.internal.expression.provider.SearchInventoryExpressionProvider",			
"com.hcl.commerce.search.internal.expression.provider.SearchByPublishedEntryOnlyExpressionProvider",
"com.hcl.commerce.search.internal.expression.provider.SearchByStorePathExpressionProvider",
"com.hcl.commerce.search.internal.expression.provider.SearchBySellerExpressionProvider",
"com.hcl.commerce.search.internal.expression.provider.SearchByCatalogExpressionProvider",
"com.hcl.commerce.search.internal.expression.provider.SearchByLanguageExpressionProvider",
"com.hcl.commerce.search.internal.expression.provider.SearchProductEntitlementExpressionProvider"],
		
"preprocessor" : [],
		
"postprocessor" : [
"com.hcl.commerce.search.internal.expression.postprocessor.SearchMainCatalogEntryViewResultQueryPostprocessor",
"com.hcl.commerce.search.internal.expression.postprocessor.SearchCatalogEntryViewPriceQueryPostprocessor",
"com.hcl.commerce.search.internal.expression.postprocessor.SearchCatalogEntryViewUserDataQueryPostprocessor" 
],
		
"responseFields" : [
"id.catentry",
"id.store",
"buyable",
"id.catalog",
"identifier.sku.raw",
"name.raw",
"name.override.raw",
"description.raw",
"description.override.raw",
"url.thumbnail",
"url.override.thumbnail",
"url.image",
"url.override.image",
"keyword.text",
"manufacturer.raw",
"id.member",
"seller.raw",
"seller_description.raw",
"type",
"prices.*",
"path.*",
"relationship.product.id",
"relationship.item.id",
"attribute.source",
"url.seo",
"url.override.seo",
"attachments.*",
"images.*",
"description.long"
]
}
}

The response should be similar to:

{
  "Profile created with name": "HCL_findCatalogEntrySKUs"
}

Repeat this step for theHCL_V2_findProductByPartNumber_Details search profile by adding seller_description.raw into the responseFields section of the existing search profile.
```
GET http://QUERY_HOST:QUERY_PORT/search/resources/api/v2/documents/profiles/HCL_V2_findProductByPartNumber_Details
```

Add seller_description.raw into the responseFields section of the existing search profile to use it as a request body for the below PUT call to override the default HCL_V2_findProductByPartNumber_Details search profile.

PUT http://QUERY_HOST:30920/search/resources/api/v2/documents/profiles/HCL_V2_findProductByPartNumber_Details

{
"profileName": "HCL_V2_findProductByPartNumber_Details",
"indexName": "product",
"query": {
"params": [
{
"DynamicKitReturnPrice": "true"
},
{
"maxRows": "50"
}
],
"queryFields": [
"identifier.sku.normalized"
],
"provider": [
"com.hcl.commerce.search.internal.expression.provider.SearchByCatalogExpressionProvider",
"com.hcl.commerce.search.internal.expression.provider.SearchByStorePathExpressionProvider",
"com.hcl.commerce.search.internal.expression.provider.SearchBySellerExpressionProvider",
"com.hcl.commerce.search.internal.expression.provider.SearchByLanguageExpressionProvider",
"com.hcl.commerce.search.internal.expression.provider.SearchInventoryExpressionProvider",
"com.hcl.commerce.search.internal.expression.provider.SearchProductEntitlementExpressionProvider"
],
"preprocessor": [],
"postprocessor": [
"com.hcl.commerce.search.internal.expression.postprocessor.SearchMainCatalogEntryViewResultQueryPostprocessor",
"com.hcl.commerce.search.internal.expression.postprocessor.SearchCatalogEntryViewPriceQueryPostprocessor",
"com.hcl.commerce.search.internal.expression.postprocessor.SearchCatalogEntryViewSKUQueryPostprocessor",
"com.hcl.commerce.search.internal.expression.postprocessor.SearchCatalogEntryViewAttributesQueryPostprocessor",
"com.hcl.commerce.search.internal.expression.postprocessor.SearchCatalogEntryViewAttachmentsQueryPostprocessor",
"com.hcl.commerce.search.internal.expression.postprocessor.SearchCatalogEntryViewImagesQueryPostProcessor",
"com.hcl.commerce.search.internal.expression.postprocessor.SearchCatalogEntryViewComponentsQueryPostprocessor",
"com.hcl.commerce.search.internal.expression.postprocessor.SearchCatalogEntryViewMerchandisingAssocQueryPostprocessor",
"com.hcl.commerce.search.internal.expression.postprocessor.SearchCatalogEntryViewUserDataQueryPostprocessor"
],
"responseFields": [
"id.catentry",
"id.store",
"id.catalog",
"buyable",
"identifier.sku.raw",
"name.raw",
"name.override.raw",
"description.raw",
"description.override.raw",
"url.thumbnail",
"url.override.thumbnail",
"keyword.text",
"manufacturer.raw",
"id.member",
"seller.raw",
"seller_description.raw",
"type",
"prices.*",
"path.*",
"relationship.item.*",
"relationship.product.id",
"relationship.component.*",
"subscription.type",
"subscription.recurring",
"url.image",
"url.override.image",
"description.long",
"attribute.source",
"kit.components",
"kit.preconfigured",
"kit.model",
"kit.URL",
"kit.default_configuration",
"kit.pdks",
"kit.parent.model",
"kit.configurable",
"kit.parent.configurable",
"associations.*",
"url.seo",
"attachments.*",
"images.*"
]
}
}

The response should resemble:

{
  "Profile created with name": "HCL_V2_findProductByPartNumber_Details"
}

Restart the Query Service container.

Test the customization to verify that the customization is successful.

In this topic, you have covered how to modify the response fields by overriding query search profiles .

Testing the Ingest profile extensions

In this topic, you will learn to test the Ingest profile extensions.

Procedure

Test the Ingest profile extensions.

The testing step is the same for both developer and runtime environments.

Build the index for the EmeraldMP store (storeId = 31).
Once the indexing has been finished and the Nifi UI shows 0/0 bytes queued data perform the following tests to validate that the response fields contain seller_description.raw when search is performed for both search profiles HCL_findCatalogEntrySKUs and HCL_V2_findProductByPartNumber_Details.
- Test 1: Validate that the response for the following search query endpoint retrieves seller_description.raw for profile HCL_findCatalogEntrySKUs .
  Execute the following GET command using a REST client.
```
GET http://QUERY_HOST:QUERY_PORT/search/resources/api/v2/products?storeId=31&seller=7000000000000007002&id=14503
```
- Test 2: Validate that the response for the following query endpoint retrieves seller_description.raw for profile HCL_V2_findProductByPartNumber_Details.
  Execute the following GET command using a REST client.
```
GET http://QUERY_HOST:QUERY_PORT/search/resources/api/v2/products?storeId=31&partNumber=MPHBA031_3103&seller=7000000000000007002
```