Search index verification

You can use the search index verification framework to check whether a search index is valid. You can define specific tasks to run, depending on if the search index is valid or not. The managed configuration uses search index verification by default to verify the managed repeater and subordinate index cores. You can use this framework to automates custom tasks based on the integrity of the search index.
The following diagram shows search index verification framework logic:
Operations logic flow chart
Where:
  1. Check operations verify the integrity of the search index by validating the data integrity of the index, or by validating the index files integrity.
  2. Optional: Upon success operations are run if all of the check operations return a success status. Typical upon success operations include tasks such as creating an index backup, or cache invalidation.
  3. Optional: Upon failure operations are run if any of the check operations return a failure status. Typical upon failure operations include tasks such as restoring an index backup, or sending error notifications to the site administrator.

Check operations can be configured with the replication handler so that every time replication occurs the check operations logic is called. Alternatively, upon success or upon failure operations can be configured with their own handler to be called directly if needed. The default operations are configured by using the /operation? handler.

Example

A typical search index verification flow goes through the following tasks:
  1. A check operation is configured to run on a repeater or on a subordinate server against the search index.
  2. The check operation runs automatically against the newly replicated index to validate the replicated index.

    Check operations can be a set of Solr queries against the index to verify the catalog data, or a sophisticated operation to validate the index segment files that verifies every indexed document. Check operations must return either a true or false status, which is then used to determine any further actions.

  3. If all check operations succeed, you might want to run upon success operations that create an index backup, trigger cache invalidation, or raise any other relevant events.

    Alternatively, if one of the check operations report a problem with the newly replicated index, you might want to run upon failure operations that rebuild the index, or restore an earlier backup, or refetch the index from the master server. In most failure cases, you might want to have upon failure operations that notify the site administrator of the problem.

You can configure check, upon success, and upon failure operations to suit your business needs. The complexity of the operations depends on the available resources, and any time constraints of the running operations.

Extension points

The search index verification framework provides extension points for you to create and configure the operations that you require.

The managed configuration uses the search index verification framework by enabling healthCheckOps and indicating the checkOps flag in the solrconfig.xml file:

<lst name="healthCheckOps">
       <str name="enable">${healthCheckOps.enable:false}</str>
       WebSphere Commerce Version 8.0.3.0 or later<str name="forceHealthCheckEveryPollInterval">${healthCheckOps.forceHealthCheckEveryPollInterval:true}</str>
       <str name="checkOps">${healthCheckOps.checkOps:com.ibm.commerce.foundation.solr.operation.SolrDoQueryCheckOperation}</str>
	<str name="uponSuccessOps">${healthCheckOps.uponSuccessOps:}</str>
	<str name="uponFailureOps">${healthCheckOps.uponFailureOps:}</str> 
</lst>

The replication handler triggers the check operation for cores in the managed configuration. On the master server, the entry point to call the operations is the core operation request handler, or your own defined custom request handler. Check operations logic is automatically triggered and run every time index replication completely downloads new index files and switches over to the new index. As a result, the upon success or upon failure operations are automatically triggered on the repeater and subordinate servers.