Tutorial: Indexing external data in HCL Commerce search

HCL Commerce Developer

In this tutorial, you add the ability to index data from a data source that is external to the HCL Commerce database. You then add Management Center support for ranking product search results based on the indexed customer ratings data. In this tutorial, your external source data is product ratings from customers.

Through customization of the indexing process, you add data to the search index to enable searching on the customer ratings data. Newly added search fields are used as ranking fields to sort the search results when you create your search rules. Search rules provide ways to manipulate the ordering and ranking of search results in starter stores, where the new ranking field is used as a sort criteria.

Note: Some extensions that are used in this tutorial contain an X_ prefix. This naming convention prevents naming conflicts between customization properties and default HCL Commerce properties.
The indexing process in HCL Commerce search is divided into two stages:
  1. Preprocessing, where the highly structured relational data is flattened and stored in summary tables.
  2. The actual indexing, where data is extracted from the flattened tables and sent to the Solr indexes.

This two-stage process provides a customization point that allows external data to be added to the index during the preprocessing stage. To configure the preprocessing stage, you use the preprocessing configuration extensions to create temporary tables to hold the external ratings data, which is used in the full index build. You must create the Java extension classes for loading the product ratings data from the XML file that contains the externally sourced data into the temporary tables.

You must customize the index building process to include data from the temporary tables into the search index. To customize this process, you configure the DIH configuration file to associate the new table data with the existing data before indexing. Once the indexing process is customized to include the external data in the search index, you add support for using the data in a sorting filter in the Management Center.

To display the ratings data in the storefront and use it in the sorting filter, the new ranking field must be added to the mediator configuration file. You configure the new ranking field to map the fields into the UserData within the CatalogNavigationViewType noun, the noun that is used for search results. With the mapping complete, the fields can be extracted for display in your store. The following diagram illustrates this process:
Ratings tutorial process diagram
Note: Workspaces are not supported when you are completing this tutorial.

Learning objectives

After completing this tutorial, you should be familiar with indexing and with the following tasks:
  • Understand and customize HCL Commerce search
  • Understand and customize the preprocessing and indexing process

Time required

Expect this tutorial to take 2 hours to complete. The tutorial can take longer if you explore concepts that are related to this tutorial.

Skill level

This tutorial is intended for advanced HCL Commerce developers responsible for creating and customizing HCL Commerce BOD services. Ensure that you are familiar with the following terms and concepts before you begin this tutorial:
  • HCL Commerce search
  • HCL Commerce services
  • XML
  • Relational databases
  • SQL

System requirements

Before you begin this tutorial, ensure that you complete the following tasks in your development environment:
  • Deploy the HCL Commerce search server.
  • Populate and build the search index.

Tutorial resources

Download and extract the following compressed file into a temporary directory in your development environment.