Building the WebSphere Commerce search index
Before you begin
- Ensure that the test server is started in Rational Application Developer.
- Ensure that your administrative server is started. For example:
- If WebSphere Commerce is managed by WebSphere Application Server Deployment Manager (dmgr), start the deployment manager and all node agents. Your cluster can also be started.
- If WebSphere Commerce is not managed by WebSphere Application Server Deployment Manager (dmgr), start the WebSphere Application Server server1.
- Completed the following task, Preprocessing the WebSphere Commerce search index data.
- Ensure that you can visit Solr over HTTP, for example:
- http://host_name/solr/Default/select?q=*%3A* Note: To index site content, do one of the following tasks:
- Set autoIndex to true in the droidConfig.xml file when crawling.
- Pass the -basePath -storeId -localename parameters to the di-buildindex utility.
- http://host_name:3737/solr/Default/select?q=*%3A*
- http://host_name/solr/Default/select?q=*%3A*
Procedure
- Complete one of the following tasks:
- Log on as a WebSphere Commerce non-root user.
- Log on with a user profile that has *SECOFR authority.
- Log on with a user ID that is a member of the Windows Administration group.
- Go to the following
directory:
- WC_installdir/bin
- WCDE_installdir\bin
- Run the index building
utility:
-
di-buildindex.bat
-instance instance_name -masterCatalogId masterCatalogId [-localename localename] [-indextype indextype] [-indexSubType indexSubType] -dbuser dbuser -dbuserpwd dbuserpwd [-fullbuild true | false] [-statusInterval statusInterval] [-storeId storeId] [-webcontentDelete true | false] [-solrConnTimeout solrConnTimeout] [-soTimeout soTimeout] [-workspace workspaceId] [-retries retries] [-passwordFile passwordFile] [-restartTime restartTime]
-
di-buildindex.sh
-instance instance_name -masterCatalogId masterCatalogId [-localename localename] [-indextype indextype] [-indexSubType indexSubType] -dbuser dbuser -dbuserpwd dbuserpwd [-fullbuild true | false] [-statusInterval statusInterval] [-storeId storeId] [-webcontentDelete true | false] [-solrConnTimeout solrConnTimeout] [-soTimeout soTimeout] [-workspace workspaceId] [-retries retries] [-passwordFile passwordFile] [-restartTime restartTime]
-
di-buildindex.bat -masterCatalogId masterCatalogId [-indextype indextype]
- Apache Derby supports only one database
connection, you must provide more parameters to build the web content
index subtype:
di-buildindex.bat -masterCatalogId masterCatalogId [-localename localename] [-indextype indextype] [-indexSubType indexSubType] [-webcontentDelete true | false] [-basePath basePath] [-storeId -storeId]
Where:- instance_name
- The name of the WebSphere Commerce instance with which you are working (for example, demo).
- The instance name is optional in WebSphere Commerce Developer.
- masterCatalogId
- Required: The ID of the master catalog (for example, 10101).If you do not know the master catalog ID, run the following SQL:
SQL: select * from catalog where IDENTIFIER='STORE_IDENTIFIER'
To find the master catalog ID for an Extended Site store:- Find the store ID:
select * from storeent where IDENTIFIER='STORE_IDENTIFIER'
- Use the storeent_id as
the store_id in the following SQL to find the catalog asset store
ID of this Extended Site store:
Where XXXXXX is the storeent_id from the SQL in step 3.a when building the search index.select * from storerel where store_id=XXXXXX and streltyp_id=-4 and relatedstore_id not in (XXXXXX)
- Get the master catalog ID:
Where YYYYYY is the relatedstore_id from step 3.b when building the search index.select * from storecat where storeent_id=YYYYYY and mastercatalog='1'
- Find the store ID:
- localename
- Optional: The locale to index.
- All
- en_US
- fr_FR
- de_DE
- it_IT
- es_ES
- pt_BR
- zh_CN
- zh_TW
- ko_KR
- ja_JP
- ru_RU
- ro_RO
- pl_PL
- indextype
- Optional: Indicates the search engine index to set up for a more
granular level of indexing. Valid values:
- CatalogEntry: Sets up the index for catalog entries in the master catalog.
- CatalogGroup: Sets up the index for categories
in the master catalog.
If you do not use the indextype parameter, both the CatalogEntry and CatalogGroup indexes are built by default.
- indexSubType
- Optional: Indicates the search engine index subtypes to set up. Valid values:
- Structured
- Sets up the index for structured content.
- Unstructured
- Sets up the index for unstructured content.
- WebContent
- Sets up the index for site content.
- Inventory
- Sets up the index for inventory data.
If you do not use the indexSubType parameter, the Structured, Unstructured, and WebContent index subtypes are built by default. If you set the indextype to be CatalogGroup, you can set the indexSubType to be Structured. You can set any indexSubType value when you set CatalogEntry to be the indextype value.
- dbuser
- Required for DB2 and Oracle databases:
The name of the user who is connecting to the database.
The user ID connecting to the database.
Optional for Derby databases.
- dbuserpwd
- Required for DB2 and Oracle databases: The password for the user who is connecting to the
database.
Alternatively, you can use the passwordFile parameter to specify the encrypted password from a file.
Optional for Derby databases.
- fullbuild
- Optional: A flag that indicates whether it is a full index build. The accepted values are either true or false. The default value is true.
- statusInterval
- Optional: The interval in milliseconds that the utility uses to
check the index building status. The default value is 10,000 milliseconds.Note: If it takes too long to index many languages, reduce the status interval to a lower value.
- basePath
- Optional: A list of directories which contains the manifest.txt file
for web content. For instance, WCDE_installdir\workspace\Stores\WebContent\Madisons\StaticContent\en_US is
for the Madisons starter store in United States English.
In Extended Sites, if different Extended Site stores have their own pages to index, multiple directories are passed into -basePath separated by a comma. In this case, the -storeId must be provided, with the order of the storeIds according to the order of the basePath.
- searchuser
- Optional: The user name of the search server. This parameter is required if WebSphere Application Security is enabled for WebSphere Commerce search.
- searchuserpwd
- Optional: The password of the search server user. This parameter is required if WebSphere Application Security is enabled for WebSphere Commerce search.
- solrConnTimeout
- Optional: The time in milliseconds that the Solr connection stays open before it times out. The default value is 100 milliseconds. Use this parameter if you encounter timeout issues when you run the di-buildindex utility.
- soTimeout
- Optional: The time in milliseconds that the socket read stays open before it times out. Use this parameter if you encounter timeout issues when you run the di-buildindex utility.
- storeId
- Optional: Used with basePath, in Extended Sites, if different Extended Site stores have their own pages to index, multiple directories are passed into -basePath separated by a comma. In this case, the -storeId must be provided, with the order of the storeIds according to the order of the basePath.
- webcontentDelete
- Optional: A flag to indicate whether WebSphere Commerce search
deletes the site content index. For instance, to delete and rebuild
the current site content index: Run the di-buildindex utility,
specify
webcontentDelete= true
, then rerun the di-buildindex utility and specifywebcontentDelete = false
.The default value is false.
- workspace
- Optional: The workspace index to build. This value is case-sensitive. If specified, the
specified workspace index is built. If not specified, the base schema index is built. The default
value is to build the base schema index. To get the workspace ID, either:
- Open the workspace in the Workspace Management tool in the Management Center. The workspace
code
is the workspace ID; or - If the workspace has an active task group, run the following SQL query:
select * from cmwsschema
, where the workspace ID is listed under theworkspace
column.
- Open the workspace in the Workspace Management tool in the Management Center. The workspace
- dbURL
- 1 The database URL the utility uses to connect to the database. If not provided, the utility constructs a database URL based on the default database value.
- retries
- Optional: Indicates the number of times the utility retries sending
a status command check to the Solr server before indicating that the
di-buildindex utility has failed. Retries might help resolve minor
issues during the index build. For example, to recover from temporary
networking errors.
You must apply the interim fix for APAR #JR49048 to use this parameter.
- numOfLangsParallel
- Optional: Indicates the number of languages to build in parallel.
For example, indicating 2 results in the first two languages being
built in parallel, followed by the next two when complete, until all
languages are built. If not specified, all languages are built at
the same time.
You must apply the interim fix for APAR #JR53029 to use this parameter.
- passwordFile
- Optional: The full path to the password.properties file
that contains the password for the user who is connecting to the database.
For instance, C:\password.properties.The password.properties file contains the following content:
Where encrypted_pwd is the password that is encrypted with the wcs_encrypt utility.dbUserPassword=encrypted_pwd
- restartTime
- The restart time to begin invalidating the cache. The input pattern
is MM/dd/yyyy_HH:mm:ss.Note: This parameter does not support Derby databases.
The utility reports the status of the indexing progress that is based on the statusInterval parameter. By default, the utility print every 10 seconds how many documents were indexed in each index, how long the utility was running, and the current indexing status. After the utility completes, it reports how many documents were successfully indexed in each index, and which index build failed.
-
- Ensure that the utility runs successfully. If the utility runs successfully, the following message is displayed in the Command window:
Data import process completed successfully with no errors.
Also, inspect the following file for errors:- WC_installdir\logs\wc-dataimport-buildindex.log
If there is any error in the wc-dataimport-buildindex.log, you can find more information from Solr_profiledir/logs/solrServer/SystemOut.log
To get more logging information, update the logging level fromINFO
toFINEST
in the WC_installdir/instances/instance_name/xml/config/dataimport/buildindex-logging.properties file:# Default global logging level, INFO .level=FINEST
# Default global logging level, INFO com.ibm.commerce.level=FINEST
You can also increase the log file size and the number of log files. For instance:# Limiting size of output file in bytes: java.util.logging.FileHandler.limit=50000000 # Number of output files to cycle through java.util.logging.FileHandler.count=2
- Restart the WebSphere Commerce Test Server.