Scenario: Initial load
After an HCL Commerce instance is created and configured, you can load the initial catalog, price, and inventory data into the HCL Commerce database.
Configuration Recommendations
During your initial data loads, you can
typically load a large amount of data if you have a large catalog. At this stage, the commerce
server is not running and the data load can use all resources available to load data into the
database. Therefore, it is important to configure the Data Load utility to make the initial load
efficient. The following configuration considerations are recommended to help improve the initial
load performance:
- Specify the data load mode as
Insert
in the wc-dataload.xml data load order configuration file. In this mode, the insert SQL statements are generated without checking whether any data exists in your database. Therefore, the performance is better than theReplace
mode because less checking is done on the database and less memory is used for the ID resolver cache. - Specify a large commit count and batch size.
- Specify a large ID resolver cache size and large JVM heap size. If you specify a large ID resolver cache, the Data Load utility does not need to resolve the ID from the database directly. But the ID resolver cache might use a significant amount of JVM heap memory. For 1 GB JVM heap size, set the ID resolver cache size to be less than 2 million to prevent a Java out of memory error.
Sample
The following code snippet is part of the sample
wc-dataload.xml data load order configuration file. This snippet specifies the
Insert
data load mode, a large commit count, and batch
size:<_config:LoadOrder commitCount="1000" batchSize="1000" dataLoadMode="Insert" >
<_config:LoadItem name="CatalogEntry" businessObjectConfigFile="wc-loader-catalog-entry.xml"
<_config:DataSourceLocation location="CatalogEntry.csv" />
</_config:LoadItem>
</_config:LoadOrder>
The
following code snippet is part of a sample wc-dataload-env.xml data load
environment configuration file that specifies the ID resolver cache
size:
<_config:IDResolver className="com.ibm.commerce.foundation.dataload.idresolve.IDResolverImpl" cacheSize="2000000" />