Propagating the WebSphere Commerce search index with the repeater

You can use the indexprop utility to propagate the WebSphere Commerce search index. It sends an HTTP request to the search index repeater to start replicating with the staging search index. This process ensures that your catalog changes are populated into the WebSphere Commerce search index in production.

The indexprop utility accepts additional parameters:

An optional index validation parameter (-queryCheckPropFile) to validate the index on the master server. Only when all index validations are successful, the index replication is triggered; otherwise, no replication occurs.
An optional parameter (-disableReplicationOnRepeaterUntilIndexPropSuccessful ) that blocks replication from the repeater while the utility is running, to synchronize all the new indexes and therefore be available at the same time for all subordinate servers to pull.

The search index repeater is used as both a master and a subordinate for search replication.

It is used as a subordinate when replicating with the staging search index, where the staging search index is the master and the repeater is the subordinate acting as a backup of the search index for production. Once the first replication is completed from staging, the repeater communicates the changes to its subordinate nodes that are in production.

The repeater then becomes the master, where all nodes from the search index cluster are configured to poll changes from the repeater on a regular pre-configured fixed-time interval. This time interval is defined in the solrconfig.xml file under replication.

Replicating between the repeater and all search index clusters in production can be automated, as the indexed data in the repeater always matches the indexed data in production. The search index repeater must be a subordinate to the staging search server and master to the production search server.

Important: The repeater must reside in Production, as it relies on the production database to perform emergency updates.

The following diagram illustrates the timing of events you must consider when indexing with staging propagation:
Timeline of events when indexing with staging propagation

Timeline of events when indexing with staging propagation

Before you begin

Ensure that the test server is stopped and that Rational Application Developer is not running.
Ensure that your administrative server is started. For example:
- If WebSphere Commerce is managed by WebSphere Application Server Deployment Manager (dmgr), start the deployment manager and all node agents. Your cluster can also be started.
- If WebSphere Commerce is not managed by WebSphere Application Server Deployment Manager (dmgr), start the WebSphere Application Server server1.
If global security is enabled, ensure that security credentials for replication are set up for all search servers.

Procedure

Complete one of the following tasks:
- Log on as a WebSphere Commerce non-root user.
- Log on with a user ID that is a member of the Windows Administration group.
If you are using this utility to replicate the search index, update the replication configuration file on the staging server to match your WebSphere Commerce search environment:
1. Open the replication.csv file for editing.
  The default location of the file is solrHome/replication.csv.
  Note: When running the UpdateSearchIndex scheduler job:
  
  The UpdateSearchIndex scheduler job does not run indexprop by default. Therefore, the replication.csv does not need to be copied to a location outside the Solr home directory.
  
  The replication.csv file should be copied to a location outside the Solr home directory. This avoids replication automatically taking place every time the UpdateSearchIndex scheduler job is run. For example, copy the replication.csv to the WC_installdir/instances/instance_name/search directory. Then pass the -solrHome value when running the indexprop command.
2. Update the replication configuration file with the following information:
  
  SearchServerName
  
  The repeater server name.
  
  SearchServerPort
  
  The repeater server port.
  
  CoreName
  
  The core name to replicate.
  
  UserName
  
  The user name for the application security-enabled subordinate server.
  
  If application security is not enabled, set this value to null.
  
  Password
  
  The password for the application security-enabled subordinate server.
  
  The password must be encrypted by the wcs_encrypt utility.
  
  If application security is not enabled, set this value to null.
  For more information on how to update the replication configuration file (replication.csv), download and extract the following archive that contains sample CSV files:
  - sdsearch_replication_samples.zip
3. Ensure that the core to replicate is configured in the solr.xml file.
Go to the following directory:
- WC_installdir/bin
- WCDE_installdir\bin
Run the indexprop utility:
To run the utility:
- indexprop.bat -masterCatalogId masterCatalogId -instance instance_name --dbuser dbuser -dbuserpwd dbuserpwd -solrHome solrHome
- ./indexprop.sh -masterCatalogId masterCatalogId -instance instance_name --dbuser dbuser -dbuserpwd dbuserpwd -solrHome solrHome
- indexprop.bat -masterCatalogId masterCatalogId -solrHome solrHome
Where:

searchServerName

The search repeater server host name.

searchServerPort

The search repeater server port.

The default value is 3737.

solrHome

The location of the Solr home directory path that contains the search index data. The value must be an absolute path. The default value is: WC_installdir/instances/instance_name/search/solr/home
Alternatively, to run the utility to perform cache invalidation on the production server:
- indexprop.bat -solrHome solrHome -masterCatalogId masterCatalogId -instance instance_name -dbuser dbuser -dbuserpwd dbuserpwd [-destdb destdb] [-destdb_user destdb_user] [-destdb_passwd destdb_passwd] [-restartTime restartTime] [-dbtype dbtype] [-destdb_schema destdb_schema] [-destdb_url destdb_url] [-passwordFile passwordFile] [-disableReplicationOnRepeaterUntilIndexPropSuccessful true|false] [-queryCheckPropFile queryCheckPropFile]
- ./indexprop.sh -solrHome solrHome -masterCatalogId masterCatalogId -instance instance_name -dbuser dbuser -dbuserpwd dbuserpwd [-destdb destdb] [-destdb_user destdb_user] [-destdb_passwd destdb_passwd] [-restartTime restartTime] [-dbtype dbtype] [-destdb_schema destdb_schema] [-destdb_url destdb_url] [-passwordFile passwordFile] [-disableReplicationOnRepeaterUntilIndexPropSuccessful true|false] [-queryCheckPropFile queryCheckPropFile]
Where:
solrHome

The location of the directory that contains the replication configuration file (replication.csv).

masterCatalogId

The ID of the master catalog (for example, 10101).

instance

The name of the WebSphere Commerce instance with which you are working (for example, demo).

dbuser

The name of the user that is connecting to the database.

The user ID connecting to the database.

dbuserpwd

The password for the user that is connecting to the database.

Alternatively, you can use the passwordFile parameter to specify the encrypted password from a file.

connectTimeout

The time, in milliseconds, before the connection times out fetching the search index from the master server. The default value is 10000.

sleepTimeout

The time, in milliseconds, before the connection times out waiting for the replication status. The default value is 10000.

dbUrl

The URL used to connect to the database.
Note: This parameter is only needed for Oracle Real Application Clusters (RAC).

dbtype

The production database type to connect to.

This parameter is needed for IBM i for performing cache invalidation on the production server.

The accepted values are DB2/OS400 or DB2/OS400ToolBox.

destdb

The production database name to connect to.

This parameter is needed for performing cache invalidation on the production server.

If the database type is DB2/OS400, specify the database name of the production server. For example, db_name.

If the database type is DB2/OS400ToolBox, specify the database host of the production server. For example, db_host.

If the database type is DB2 or Oracle, specify the database host, database server port, and database name of the production server. For example, db_host:db_port/db_name.

destdb_user

The production database user name to connect with.

This parameter is needed for performing cache invalidation on the production server.

destdb_passwd

The production database password to connect with.

This parameter is needed for performing cache invalidation on the production server.

Alternatively, you can use the passwordFile parameter to specify the encrypted password from a file.

destdb_schema

The production database schema to connect to.

This parameter is optional for performing cache invalidation on the production server.

destdb_url

The production database URL to connect to.

This parameter is optional for performing cache invalidation on the production server.
Note: This parameter is only needed for Oracle Real Application Clusters (RAC).

restartTime

The restart time to begin invalidating the cache on the production server. That is, the time stamp when the last stagingprop command starts running.

This parameter is needed for performing cache invalidation on the production server.

Accepted values are long values that return milliseconds in the following format: MM/dd/yyyy_HH:mm:ss.

passwordFile

Optional: The full path to the password.properties file that contains the password for the user who is connecting to the database, and the production database password to connect with. For example, C:\password.properties.
The password.properties file contains the following content:
```
dbUserPassword=encrypted_pwd
prod.dbUserPassword=encrypted_pwd
```
Where encrypted_pwd is the password that encrypted by the wcs_encrypt utility.
disableReplicationOnRepeaterUntilIndexPropSuccessful

Indicates whether to disable replication on the repeater so that subordinate servers cannot pull until index propagation is successful. Replication is then enabled upon completing successfully.

The default value is false.

queryCheckPropFile

The fully qualified path to a test query properties file used to validate the search index before propagating. Replication is aborted if any of the test queries fail.

For more information, see Query check properties file.
Ensure that the utility runs successfully.
Inspect the following log file for errors:
- WC_installdir/logs/wc-indexprop.log
To get more logging information, update the logging level from INFO to FINEST in the WC_installdir/instances/instance_name/xml/config/dataimport/indexprop-logging.properties file:
```
# Default global logging level, INFO
com.ibm.commerce.level=FINEST
```
After you propagate the WebSphere Commerce search index, you can verify the changes in the storefront.