Index-building topologies and scenarios

WebSphere Commerce search indexes are created separately based on a specific master catalog. Deploy the WebSphere Commerce search index, then separately manage and rebuild each index to refresh its data.

Search indexing methods

A full reindexing rebuilds the entire search index, while a delta reindexing performs only incremental updates on the existing operational search index.

Important: You must periodically fully rebuild the search index to ensure that the index size on the file system remains manageable. That is, when only delta reindexing is performed, the index files that marked for deletion continue to accumulate. When a full reindexing is performed, an optimize request is passed in that fully deletes the index files identified from previous delta reindexes.

Before you deploy your index, consider the index-building scenarios, depending on your WebSphere Commerce search environment.

The following topologies are typical:

A production server with a dedicated index-building machine.
A staging and clustered search server.

It is recommended that you use the same operating system across all search servers (master search server, subordinate servers, and search server cluster).

Search index types

WebSphere Commerce search contains the following search index types to suit your business and search requirements:

Catalog entry index

A search index for catalog entries in both master and sales catalog.

The catalog entry search index contains the following content:

Structured content: Structured content includes items in the product catalog and delivers search results based on items that are sold in your store.
Unstructured content: Unstructured site content includes documents that do not adhere to a specific data model, such as product attachments contained in various formats. For example, content such as user manuals and warranty information are considered unstructured content, as its elements, construction, and organization are typically unknown and can vary depending on its file type.
Site content: Site content includes HTML and other site files from WebSphere Commerce starter stores. It is fetched and crawled by the site content crawler.

Catalog group index

A search index for categories in both master and sales catalog.

WebSphere Commerce search contains the following search index subtypes, which keeps data in a separate core for performance reasons:

Inventory

The inventory index, a separate index that contains index data, is an extension of the product index. For accurate inventory status, you can refresh the inventory index more frequently than the product index.

Important:

Non-ATP and DOM inventory systems are supported by index inventory data in WebSphere Commerce Search. ATP, External, and No-inventory types are not supported by the Search inventory index.
The filterCache and documentCache are required when an extension index such as Inventory exists in WebSphere Commerce search. For more information, see WebSphere Commerce search performance tuning.

Price

Sets up a subindex for price data. Prices are indexed using Index Load, as it can populate a large amount of data into a separate extension index faster than the Catalog Entry index can index price data. For more information, see Index Load.

For more information about extension indexes, see WebSphere Commerce search extension indexes.

Index-building topologies and scenario examples

The following sample topologies are available for small to large index sizes:

Important: In all scenarios, it is recommended to only perform indexing in an authoring or staging environment. That is, an environment that is used for quality assurance (QA) purposes, and if required, containing access to the production database. Otherwise, the runtime search performance or index integrity can be affected by indexing within a production environment.

Small or medium index size: WebSphere Commerce production server with a dedicated index-building machine

A dedicated index-building machine, which is known as the master, is used to build the index. The index is then replicated to other search machines, which are known as targets. The runtime search performance is not affected during index-building.

Note: Building the search index can still be a timely process, and might occasionally lead to inconsistencies between the database and search index. However, the runtime search performance is not affected during index-building due to the offset of the processing load to another machine.

The following diagram illustrates a typical WebSphere Commerce search deployment for a small or medium index size:
Small or medium index size deployment

Recommended: Large index size: Staging propagation and clustered search servers

The index-building is performed on the WebSphere Commerce staging database and search staging machine. Business users can test their new data in the staging database with the updated index. After successfully completing their tests, the data is propagated from the staging database to the production database, and the index is replicated from the search staging machine to the search production machines using the master indexing server. This is the recommended approach, as this option imposes the least amount of risk to the production environment and provides a more flexible environment for making incremental changes.

The following diagram illustrates a typical WebSphere Commerce search deployment for a large index size: Feature Pack 4

For more information, see Indexing with staging propagation.