Propagating the WebSphere Commerce Search index with the repeater

You can use the indexprop utility to propagate the WebSphere Commerce search index. It sends an HTTP request to the search index repeater to start replicating with the staging search index. This process ensures that your catalog changes are populated into the WebSphere Commerce Search index in production.

The search index repeater is used as both a master and a subordinate for search replication.

It is used as a subordinate when replicating with the staging search index, where the staging search index is the master and the repeater is the subordinate acting as a backup of the search index for production. After the first replication is completed from staging, the repeater communicates the changes to its subordinate nodes that are in production.

The repeater then becomes the master, where all nodes from the search index cluster are configured to poll changes from the repeater on a regular preconfigured fixed-time interval. This time interval is defined in the solrconfig.xml file under replication.

Replicating between the repeater and all search index clusters in production can be automated, as the indexed data in the repeater always matches the indexed data in production. The search index repeater must be a subordinate to the staging search server and master to the production search server.

Important: The repeater must reside in Production, as it relies on the production database to perform emergency updates.
The indexprop utility accepts optional parameters to validate the index on the master server, and block replication from the repeater while the utility is running:
  • An optional index validation parameter (-queryCheckPropFile) to validate the index on the master server. Only when all index validations are successful, the index replication is triggered; otherwise, no replication occurs.
  • An optional parameter (-disableReplicationOnRepeaterUntilIndexPropSuccessful ) that blocks replication from the repeater while the utility is running, to synchronize all the new indexes and therefore be available at the same time for all subordinate servers to pull.
The following diagram illustrates the timing of events you must consider when indexing with staging propagation:
Timeline of events when indexing with staging propagation

Before you begin

  • WebSphere Commerce DeveloperEnsure that the test server is stopped and that Rational Application Developer is not running.
  • Ensure that your administrative server is started. For example:
    • If WebSphere Commerce is managed by WebSphere Application Server Deployment Manager (dmgr), start the deployment manager and all node agents. Your cluster can also be started.
    • If WebSphere Commerce is not managed by WebSphere Application Server Deployment Manager (dmgr), start the WebSphere Application Server server1.
  • If global security is enabled, ensure that security credentials for replication are set up for all search servers.

Procedure

  1. Complete one of the following tasks:
    • LinuxAIXLog on as a WebSphere Commerce non-root user.
    • WindowsLog on with a user ID that is a member of the Windows Administration group.
  2. If you are using this utility to replicate the search index, update the replication configuration file on the staging server to match your WebSphere Commerce Search environment:
    1. Open the replication.csv file for editing.
      The default location of the file is solrHome/replication.csv.
      Note: When running the UpdateSearchIndex scheduler job:
      • The UpdateSearchIndex scheduler job does not run indexprop by default. Therefore, the replication.csv does not need to be copied to a location outside the Solr home directory.
      • The replication.csv file should be copied to a location outside the Solr home directory. This avoids replication automatically taking place every time the UpdateSearchIndex scheduler job is run. For example, copy the replication.csv to the WC_installdir/instances/instance_name/search directory. Then, pass the -solrHome value when running the indexprop command.
    2. Update the replication configuration file with the following information:
      SearchServerName
      The subordinate server name.
      SearchServerPort
      The subordinate server port.
      CoreName
      The core name to replicate.
      UserName
      The user name for the application security-enabled subordinate server.

      If application security is not enabled, set this value to null.

      Password
      The password for the application security-enabled subordinate server.

      The password must be encrypted by the wcs_encrypt utility.

      If application security is not enabled, set this value to null.

      For more information on how to update the replication configuration file (replication.csv), download and extract the following archive that contains sample CSV files:
    3. Ensure that the core to replicate is configured in the solr.xml file.
  3. Go to the following directory:
    • LinuxAIXWindowsWC_installdir/bin
    • WebSphere Commerce DeveloperWCDE_installdir\bin
  4. Run the indexprop utility:
    • Windows indexprop.bat -solrHome solrHome -masterCatalogId masterCatalogId -instance instance_name -dbuser dbuser -dbuserpwd dbuserpwd [-destdb destdb] [-destdb_user destdb_user] [-destdb_passwd destdb_passwd] [-restartTime restartTime] [-dbtype dbtype] [-destdb_schema destdb_schema] [-destdb_url destdb_url] [-passwordFile passwordFile] [-disableReplicationOnRepeaterUntilIndexPropSuccessful true|false] [-queryCheckPropFile queryCheckPropFile]
    • LinuxAIX./indexprop.sh -solrHome solrHome -masterCatalogId masterCatalogId -instance instance_name -dbuser dbuser -dbuserpwd dbuserpwd [-destdb destdb] [-destdb_user destdb_user] [-destdb_passwd destdb_passwd] [-restartTime restartTime] [-dbtype dbtype] [-destdb_schema destdb_schema] [-destdb_url destdb_url] [-passwordFile passwordFile] [-disableReplicationOnRepeaterUntilIndexPropSuccessful true|false] [-queryCheckPropFile queryCheckPropFile]
    Where:
    solrHome
    The location of the directory that contains the replication configuration file (replication.csv).
    masterCatalogId
    The ID of the master catalog (for example, 10101).
    instance
    The name of the WebSphere Commerce instance with which you are working (for example, demo).
    dbuser

    DB2The name of the user that is connecting to the database.

    OracleThe user ID connecting to the database.

    dbuserpwd
    The password for the user that is connecting to the database.
    Alternatively, you can use the passwordFile parameter to specify the encrypted password from a file.
    connectTimeout
    The time, in milliseconds, before the connection times out fetching the search index from the master server. The default value is 10000.
    sleepTimeout
    The time, in milliseconds, before the connection times out waiting for the replication status. The default value is 10000.
    dbUrl
    The URL used to connect to the database.
    Note: This parameter is only needed for Oracle Real Application Clusters (RAC).
    dbtype
    The production database type to connect to.
    This parameter is needed for IBM i for performing cache invalidation on the production server.
    The accepted values are DB2/OS400 or DB2/OS400ToolBox.
    destdb
    The production database name to connect to.
    This parameter is needed for performing cache invalidation on the production server.
    If the database type is DB2/OS400, specify the database name of the production server. For example, db_name.
    If the database type is DB2/OS400ToolBox, specify the database host of the production server. For example, db_host.
    If the database type is DB2 or Oracle, specify the database host, database server port, and database name of the production server. For example, db_host:db_port/db_name.
    destdb_user
    The production database user name to connect with.
    This parameter is needed for performing cache invalidation on the production server.
    destdb_passwd
    The production database password to connect with.
    This parameter is needed for performing cache invalidation on the production server.
    Alternatively, you can use the passwordFile parameter to specify the encrypted password from a file.
    destdb_schema
    The production database schema to connect to.
    This parameter is optional for performing cache invalidation on the production server.
    Oracledestdb_url
    OracleThe production database URL to connect to.
    OracleThis parameter is optional for performing cache invalidation on the production server.
    Note: This parameter is only needed for Oracle Real Application Clusters (RAC).
    restartTime
    The restart time to begin invalidating the cache on the production server. That is, the time stamp when the last stagingprop command starts running.
    This parameter is needed for performing cache invalidation on the production server.
    Accepted values are long values that return milliseconds in the following format: MM/dd/yyyy_HH:mm:ss.
    passwordFile
    Optional: The full path to the password.properties file that contains the password for the user who is connecting to the database, and the production database password to connect with. For example, C:\password.properties.
    The password.properties file contains the following content:
    
dbUserPassword=encrypted_pwd
prod.dbUserPassword=encrypted_pwd
    Where encrypted_pwd is the password that encrypted by the wcs_encrypt utility.
    disableReplicationOnRepeaterUntilIndexPropSuccessful
    Indicates whether to disable replication on the repeater so that subordinate servers cannot pull until index propagation is successful. Replication is then enabled upon completing successfully.
    The default value is false.
    queryCheckPropFile
    The fully qualified path to a test query properties file used to validate the search index before propagating. Replication is aborted if any of the test queries fail.
    For more information, see Query check properties file.
  5. Ensure that the utility runs successfully.
    Inspect the following log file for errors:
    • LinuxAIXWindowsWC_installdir/logs/wc-indexprop.log
    To get more logging information, update the logging level from INFO to FINEST in the WC_installdir/instances/instance_name/xml/config/dataimport/indexprop-logging.properties file:
    
# Default global logging level, INFO
com.ibm.commerce.level=FINEST

    After you propagate the WebSphere Commerce Search index, you can verify the changes in the storefront.