HCL Commerce Version 9.1.12.0

Troubleshooting: indexing fails on Price Stage 1a (Database)

Indexing can fail when your catalog dataset is large, for example: 200,000+ catalog entries with multiple price lists. Distributed File System (DFS) configurations are particularly susceptible to this failure mode.

Problem

This failure happens at a particular stage, Price Stage 1a (Database), Find Product Prices, with an error message similar to the following
Price Stage 1a (Database), Find Product Prices"
...
OFFSET 0 ROWS FETCH NEXT 1000000 ROWS ONLY",
"message": "java.io.IOException: java.io.IOException: Could not obtain next record from ResultSet"

Solution

To avoid this error, reduce the SQL scroll page size and bucket size.

  1. Navigate to NiFi Flow > auth.reindex - Price Stage 1a (Find Product Prices) .
  2. Right click and select Variables from the drop-down menu.
  3. Change the scroll.page.size value to 100000 (the default is 1000000), and the scroll.bucket.size value to 1000 from its default of 2000.

The indexing process no longer halts at Price Stage 1a (Database), Find Product Prices.

For more information on these parameters and process tuning in general, see Optimizing index build and overall flow.