Sharding input properties file

The sharding input properties file is a file that you create that is passed in to the di-parallel-process utility when indexing shards.

The sharding input properties file contains the following sections and properties:

System properties

System properties are shared by other utilities. For example, passwords that are common among utilities.

PasswordFileDir: The location of a password.properties file that you create that contains passwords encrypted by the wcs_encrypt utility.

Database properties

Database properties are used to establish database connections with the database server. All database properties start with the Database keyword, followed by the property name, which is separated by a period.

The following list shows the database properties:

Database.dbType: The database type. The accepted values are db2 or oracle.
Database.dbName: The database name.
Database.dbHost: The remote database host name.
Database.dbServerPort: The remote database port number.
Database.dbUser: The database user name.
Database.dbSchemaName: The database schema name. This value is typically the uppercase version of the database user name.
Database.jdbcURL: The database JDBC URL.
Database.jdbcDriverLocation: The database JDBC driver location.
Database.jdbcDriverType: The database JDBC driver type.

Global preprocessing and indexing properties

Global properties are used for preprocessing and indexing by all shards, if not otherwise overwritten by the shard itself. All global properties start with the Global keyword, followed by the property name, which is separated by a period.

Global properties can be grouped into the following categories: server properties, preprocessing properties, and indexing properties.

The following list shows the global server properties:

Global.index-server-name: The search server host name.
Global.index-server-port: The search server port number.
Global.index-server-user: The search server user name.

The following list shows the global preprocessing properties:

Global.preprocessing-locale

A single preprocessing locale value to be preprocessed. The default value is All, where all locales are preprocessed.

Global.auto-preprocess

Automatically preprocess all shards.

Global.preprocessing-drop-temp-table

Indicates whether to drop tables when preprocessing the search index.

Passing in a value of false uses a TRUNCATE statement on the tables.

The default value is true, which uses a DROP statement on the tables.

Note: This parameter supports only DB2 9.7 or later, or Oracle databases.

Global.preprocessing-truncate-table-sql

Overwrites the default TRUNCATE table statement.

The following list shows the global indexing properties:

Global.auto-index: Automatically start indexing shards after they are preprocessed.
Global.create-core: Create the shard core in the solr.xml file, if it does not exist.
Global.unload-core: Unload the shard core from the solr.xml file, if it does not exist.
Gloabl.index-status-Interval: The time interval to monitor the indexing process. The default value is 10000.
Global.auto-merge-index: Automatically merge the index into the master server after all shard indexing is complete.

Master search server properties

There is typically only one set of master properties. They start with the keyword Master, followed by either the common keyword or the locale name (for example, en_US), followed by the property name, which is separated by a period.

The master properties must include at least the master-specific properties. Only the global server properties can be overwritten in the master properties. Otherwise, if not redefined, the global properties are used.

Note: The following properties use the en_US locale with shards A and C as a sample. Configure your sharding input properties file according to the locale and shard types that you are using.

Master.en_US.structured-index-core-name
Master.en_US.unstructured-index-core-name: The master index cores where all shard data is merged.

Horizontal Shard properties

There is typically a set of horizontal shards properties for each horizontal shard that is created by the setupSearchIndex utility. The horizontal shard properties start with the Shard keyword, followed by the shard name, followed by either the common keyword or the locale name, followed by the property name, all separated by periods.

The horizontal shard properties must include at least the horizontal shard's specific properties. All global properties can be overwritten in the horizontal shard, except for auto-merge-index. If no properties are redefined, the global properties are used.

The following list shows the horizontal shard-specific properties, using shard A as a sample:

Shard.A.common.preprocessing-dir: The path to the preprocessing XML files on WebSphere Commerce where the di-preprocess utility run. This property is common and cannot be overwritten.
Shard.A.common.preprocessing-start-range-value
Shard.A.common.preprocessing-end-range-value: The horizontal shard range start and end values.
Shard.A.common.configuration-type: The shard type to configure. For example, horizontal or vertical. When there is a combination of horizontal and vertical shard types, this property must be set so. In this case, horizontal.
Shard.A.en_US.unstructured-index-core-name
Shard.A.en_US.unstructured-index-core-dir
Shard.A.en_US.structured-index-core-name
Shard.A.en_US.structured-index-core-dir: The shard unstructured and structured core names and directories.

Vertical Shard properties

There is typically a set of vertical shards properties for each vertical shard that is created by the setupSearchIndex utility. The vertical shard properties start with the Shard keyword, followed the shard name, either the common keyword or the locale name, followed by the property name, all separated by periods.

The vertical shard properties must include at least the vertical shard-specific properties. Only the global preprocessing properties can be overwritten in the vertical shard. Otherwise, if not redefined, the global properties are used.

The following list shows the vertical shard-specific properties, using shard C as a sample:

Shard.C.common.preprocessing-dir: The path to the preprocessing XML files on WebSphere Commerce where the di-preprocess utility run. This property is common and cannot be overwritten.
Shard.C.common.configuration-type: The shard type to configure. For example, horizontal or vertical. When there is a combination of horizontal and vertical shard types, this property must be set so. In this case, vertical.

Overriding properties

To override a shard or master global property, use the following convention to overwrite it at the common level:


Master.common.index-server-name=localhost
Shard.A.common.index-server-name=localhost

To override a shard or master global property, use the following convention to overwrite it at the locale level:


Master.en_US.index-server-name=localhost
Shard.A.en_US.index-server-name=localhost

Sample properties file

You can use the following sample properties file for reference: di-parallel-process.zip.

The sample uses the en_US locale with shards A, B, and C. It contains the following files:

di-parallel-process-FEP8-linux-oracle.properties: The sample sharding input properties file for a Linux operating system with Oracle.
password.properties: The sample password properties file, referenced by the sample sharding input properties file. It contains passwords encrypted by the wcs_encrypt utility.