WebSphere Commerce Search interactions

WebSphere Commerce Search includes several key service areas, including the server architecture and programming model.

Search server architecture

A direct-access path to site navigation and the search server exists on the Solr search server. This server consists of a set of REST services, a search runtime framework that reuses the current search programming model, and a set of WebSphere Commerce foundation services that also provide access to the WebSphere Commerce database.

The following diagram shows an overview of the search server architecture:
Search interaction diagram

Where:

The REST service mapper is an Apache Wink application that uses configurations to map a REST URL to a resource handler. This handler then calls the embedded search run time.
The search run time is a pluggable programming framework that allows search expressions to be run by using properties that are defined in a configuration-based search profile. Once a search expression is composed, the runtime then forwards the request to the embedded Solr run time for processing.
The foundation service is the WebSphere Commerce core run time that can provide basic server services, such as the JDBC Query Service to access the WebSphere Commerce database.

Example

When shoppers enter a search query in the storefront, the SearchSetup.jspf file calls the wc-rest-resourceconfig.xml file by passing it the URL with the search query:


<wcf:rest var="catalogNavigationView1" url="${searchHostNamePath}$searchContextPath}/store/${WCParam.storeId}/productview/${restType}">

The wc-rest-resourceconfig.xml file maps the URL (request) to a REST resource name (for example, productView) and the associated profiles to be used for the search operation:


<Resource name="productview"
  <GetUri uri="store/{storeId}/productview/{partNumber}"
        description="Get product by part number"
        searchProfile="IBM_findProductByPartNumber_Details,IBM_findProductByPartNumber_Summary"/>

The Apache wink runtime then turn it into a REST request. A Java resource handler is used to handle the processing.

Wink uses Java annotation to convert the name-value pair in the URL into a more usable form. For example, in the ProductViewHandler:


@Path("store/{storeId}/productview")
@Description("This class provides RESTful services to get the ProductView details.")
@Encoded
public class ProductViewHandler extends AbstractSearchResourceHandler {

Where the store/{storeId}/productview/{partNumber} URL is mapped to the following method in bold:


@GET
  @Path(BY_PART_NUMBER)
  @Produces({ APPLICATION_JSON, APPLICATION_XML, APPLICATION_XHTML_XML,
      APPLICATION_ATOM_XML})
  @Description("Gets products by part number.")
  @AdditionalParameterList({
      @ParameterDescription(description = PARAMETER_ASSOCIATION_TYPE_DESCRIPTION, required = false, name = PARAMETER_ASSOCIATION_TYPE, multipleValue = true),
      @ParameterDescription(description = PARAMETER_CATALOG_ID_DESCRIPTION, required = false, name = PARAMETER_CATALOG_ID),
      @ParameterDescription(description = PARAMETER_CURRENCY_DESCRIPTION, required = false, name = PARAMETER_CURRENCY, valueProviderClass = CurrencyIdProvider.class),
      @ParameterDescription(description = PARAMETER_ENTITLEMENT_CHECK_DESCRIPTION, typeClass=Boolean.class, required = false, name = PARAMETER_CHECK_ENTITLEMENT),
      @ParameterDescription(description = PARAMETER_ATTACHMENT_FILTER_DESCRIPTION, required = false, name = PARAMETER_ATTACHEMENT_FILTER) })
  @ResponseSchema(valueSamplePath = "ProductViewHandler/productdetails.json", responseCodes = {
      @ResponseCode(code = 200, reason = RESPONSE_200_DESCRIPTION),
      @ResponseCode(code = 400, reason = RESPONSE_400_DESCRIPTION),
      @ResponseCode(code = 401, reason = RESPONSE_401_DESCRIPTION),
      @ResponseCode(code = 403, reason = RESPONSE_403_DESCRIPTION),
      @ResponseCode(code = 404, reason = RESPONSE_404_DESCRIPTION),
      @ResponseCode(code = 500, reason = RESPONSE_500_DESCRIPTION) })
  public Response findProductByPartNumber(
      @PathParam(PARAMETER_STORE_ID) @ParameterDescription(description = PARAMETER_STORE_ID_DESCRIPTION, 
                            valueProviderClass = StoreIdProvider.class, required = true) String storeId,
      @PathParam(PART_NUMBER) @ParameterDescription(description = "The product part number.") String partNumber) {

The handler then calls the embedded search runtime code, which calls the wc-search.xml file by using the search profile names to look up the provider and preprocessors to call. This method is similar to a factory assembly line, where each business component can contribute its own piece of the search expression into the main query. After the search run time constructs the final Solr query, it calls Solr and processes the output by using postprocessors that are defined in the search profile. Then, the REST handler outputs the response.

The search expression is represented by the SearchCriteria Object, which contains a set of control parameters in the form of name-value parameter maps.

For example, the following snippet from the trace.log file shows that the SearchCriteria is processed by the SolrSearchEDismaxQueryPreProcessor:


SolrSearchEDi > com.ibm.commerce.foundation.internal.server.services.search.query.solr.SolrSearchEDismaxQueryPreProcessor 
invoke(SelectionCriteria, Object) ENTRY Search profile: 'IBM_findProductsBySearchTerm', Control parameters: 
'_wcf.search.exclude.type=[],' 'DynamicKitReturnPrice=[true],' '_wcf.search.currency=[USD],' '_wcf.search.manufacturer=[],
' '_wcf.search.runAsId=[-1002],' '_wcf.search.contract=[10001],' '_wcf.search.catalog=[10052],''_wcf.search.catalog=[10052],
''_wcf.search.catalog=[10052],' '_wcf.search.internal.response.format=[json],'

After a search expression is composed, the run time forwards the request to the embedded Solr run time for processing.

If there are any parameters that you want to pass in from the storefront, but do not want to specify them in the findProductByPartNumber method for example, you can add it to the control parameter mapper in the wc-component.xml file. This file maps from the name-value pair into the control parameter in the SearchCriteria object.

Search programming model

The search runtime framework is driven by a programming pattern that is similar to a factory assembly line, where each business component can contribute its own piece of the search expression into the main query, which is then run against the Solr search engine.

The search runtime programming model uses POJO and raw data that is returned from the search server to perform simple name-value pair mapping:
Search programming model

The search run time contains two main components: the search expression provider and the search expression processor:

Search Expression Provider

Depending on the nature of the request, that is, the search profile, other business components might get involved, such as Marketing for search-based merchandising rules, or contracts for entitlement. Each business component is responsible for contributing a portion of the search expression, which gets combined with the main search expression generated by the Search REST services.

As a result, the consolidated search expression is run by the search processor. A factory class, SolrRESTSearchExpressionProvider, manages all of these expression providers at run time. The implementation of this search expression provider factory class is defined in the SearchExpressionProviderFactory.properties file on the search server.

The default SolrRESTSearchExpressionProvider performs the following high-level steps in this order:

Validates that a search profile is given by calling SolrSearchProfileNameValidator and immediately stops if none is provided.
Validates the corresponding index name by calling SolrSearchIndexNameValidator and immediately stops if invalid.
Validates the corresponding workspace information by calling SolrSearchWorkSpaceValidator and sets the information to let the processor know.
Validates the search expression to ensure that the query expression is not empty and does not contain special characters by calling SolrSearchExpressionValidator.
Creates a list of search query expression providers that are defined in the search profile where each provider contributes a portion of the search expression.

Search Expression Processor

A search processor is the central processing unit for integrating with the search engine. Its responsibility is to run the Solr expression, based on the search profile attributes, and capture the response from the search engine.

The SolrRESTSearchExpressionProcessor is a default implementation of the expression processor that is defined in the SearchExpressionProcessorFactory.properties file on the search server.

The default SolrRESTSearchExpressionProcessor composes the final Solr expression and sets up all the required bootstrap settings for running in the Solr search run time. The result set is then reformatted into a JSON object to be returned to the caller.

The expression processor performs the following high-level steps in the following sequence:

Injects a debug parameter option into the SolrQuery object by calling SolrSearchDebugQueryPreprocessor.
Generates a list of index fields to be included in the search result set by calling SolrSearchResultFieldQueryPreprocessor.
Includes a relevancy score and extra tracing information into the SolrQuery object by calling SolrSearchPreviewQueryPreprocessor.
Enables spell correction and includes its associated parameter options into the SolrQuery object by calling SolrSearchSpellCorrectionQueryPreprocessor.
Enables highlighter and includes its associated parameter options into the SolrQuery object by calling SolrSearchHighlighterQueryPreprocessor.
Injects pagination parameter options into the SolrQuery object by calling SolrSearchPaginationQueryPreprocessor.
Injects sorting parameter options into the SolrQuery object by calling SolrSearchSortingQueryPreprocessor.
Composes a list of facet fields, along with the proper settings, to be included in the search result set by calling SolrSearchFacetQueryPreprocessor.
Injects the dismax query-related expression by calling SolrSearchEDismaxQueryPreProcessor as eDismax handler must occur before the main query because the boost query control parameter might need to be removed.
Injects the response format parameter option by calling SolrSearchResponseFormatQueryPreprocessor.
Injects the main search query string into the SolrQuery object by calling SolrSearchMainQueryPreprocessor.
Injects extra custom SolrQuery parameters into the final Solr query expression by calling SolrSearchCustomQueryPreprocessor.
Creates a list of search query preprocessors that are defined in the search profile to allow modifications to the SolrQuery object right before it is sent to Solr for processing.
Issues a request against Solr to process the SolrQuery object.
Creates a list of search query postprocessors that are defined in the search profile to allow modifications to the physical data object, SearchResponse, right after the QueryResponse is returned from the Solr server.

A query preprocessor is used to modify the SolrQuery object right before it is sent to the Solr server for processing.

A query postprocessor is used to modify the QueryResponse object that is returned from the Solr server.