Configuring the data load order

0

Nothing is committed until the load item finishes processing all of its input data. If an exception occurs, the records that are successfully processed can be committed. If the exception is an SQL exception, the utility rolls back all of the data that is processed after the last committed record. If a different type of exception occurs, the utility rolls back only the data that is not successfully processed. The load operation commits any successfully processed records.

N

Where N is a positive integer value. The commit count specifies how many lines of records that are processed when it calls the database commit. The default value is 1.

You can include a configuration property "rollbackForAnyException" to control what data the utility rolls back when an exception occurs. When the property is set to true, the utility rolls back all data that is processed after the last committed record whenever an exception occurs. This roll back occurs regardless of the exception type. You can set this property within the configuration for each load item or within the parent <_config:LoadOrder> element to have the property setting apply to all load items.

Insert

All data is inserted into the database. The utility generates insert SQL statements. This mode is recommended for initial loading of data. If there are any delete flags in your data file, the flags are ignored.

In insert mode, you can specify a primary keyrange to use when the object does not exist in the database and it requires a new generated key. Specify the value within the <_config:BusinessObjectMediator> element. For example:

startKey="100001" endKey="200001"

The data writers that are supported in the insert mode:

• JDBC data writer
• Native load data writer

Note: Running the Data Load utility in insert mode can improve the performance of loading initial loads of large amounts of data. However, when you run the Data Load utility in insert mode the utility does not check your database before the utility loads your input data. The utility does not determine whether the data objects that you are creating exist in your database before the utility attempts to load and create the data objects. This behavior can cause the load process to fail when the utility attempts to create data objects that exist in your database.

For example, if your input file is a CSV file that contains a line that creates a product and a product description. If the file also contains a second line that loads the same product with a description in a different language, the load process fails. Since the utility creates the product and description in the first line, when the utility encounters the second line, the utility attempts to create the product again. Since the product exists, the load fails. To load this product and description data, you can use one of the following methods:

• Run the Data Load utility in replace mode.
• Run the Data Load utility in insert mode, but load your product and description information separately. Use one input file to load your product information, and a different input file to load all of your descriptions.

For more information about configuring the Data Load utility to run in insert mode, see Scenario: Initial load.

Replace

Default: All data is replaced in the database. The utility generates insert, update, or delete SQL statements based on the data. Replace mode replaces existing data that is contained in the database with the input data. That is, if some column information is not in the input data, the column value is updated to null or the default value if any. For example:

• If one record (line) in your data file represents a new object, it is inserted.
• If the object is in the database already, it is replaced.
• If there is a flag in the data to indicate that this object is to be deleted, the object is deleted.

In replace mode, do not specify a primary keyrange value as it can result in key conflicts within the database.

The data writers that are supported in the replace mode:

• JDBC data writer
• Native load data writer

Note: You can prevent accidentally replacing information in the database with null data. Modify the original input data that is used in the initial load when you are replacing a subset of the original data. Do not enter empty fields in your source file, unless you want the fields to contain null data in the database.

For more information about configuring the Data Load utility to run in replace mode, see Scenario: Delta load.

Update

Specified catalog entry and catalog entry description data is updated in the database. The utility compares the catalog entry data in the input file with the corresponding data for the catalog entries in the database. The utility then replaces or adds the new or changed data for only the columns that are specified in the input file. All other columns remain unchanged.

The update mode supports only the JDBC data writer.

Note: The Data Load utility can run in update mode for loading only catalog entry or catalog entry description information.

For more information about configuring the Data Load utility to run in update mode, see Scenario: Catalog entry update load.

Delete

All data that is identified in the input file that is in the database is deleted from the database. The utility ignores the value in the delete column or element in an input file in this mode. The utility generates the delete SQL statements that are needed to delete the objects in the input file. When you run the utility in this mode, only the JDBC data writer is supported in this mode.

Note: If your site uses WebSphere Commerce search, the delta search index might not rebuild correctly when you delete some catalog objects with the Data Load utility in delete mode. When you delete a child object of a catalog entry or category with the utility in delete mode, both the child and parent objects are removed from the delta search index rebuild. This removal can cause the parent catalog entry or category to no longer be indexed or display correctly in the storefront.

Use the utility in replace mode to delete catalog objects when your site uses WebSphere Commerce search. To delete objects with the utility in replace mode, include the value 1 for the Delete column of an object in your input file. If you do decide to delete catalog objects with the utility in delete mode, run a full index rebuild after the load operation completes.

<_config:LoadItem name="CatalogGroup" businessObjectConfigFile=
"wc-loader-catalog-group.xml" >

<_config:DataSourceLocation location="CatalogGroups.xml"/>

<?xml version="1.0" encoding="UTF-8" ?> 
<_config:DataLoadConfiguration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
xsi:schemaLocation="http://www.ibm.com/xmlns/prod/commerce/foundation/config 
../../../../xml/config/xsd/wc-dataload.xsd" 
xmlns:_config="http://www.ibm.com/xmlns/prod/commerce/foundation/config">
  <_config:DataLoadEnvironment configFile="wc-dataload-env.xml" />
  <_config:LoadOrder commitCount="100" batchSize="1" dataLoadMode="Replace">
    <_config:property name="firstTwoLinesAreHeader" value="true" />
    <_config:LoadItem name="CatalogGroup" businessObjectConfigFile="wc-loader-catalog-group.xml">
      <_config:property name="validateCatalogGroup" value="true" />
      <_config:DataSourceLocation location="CatalogGroups.csv" />
    </_config:LoadItem>
    <_config:LoadItem name="CatalogEntry" businessObjectConfigFile="wc-loader-catalog-entry.xml">
      <_config:DataSourceLocation location="CatalogEntries.csv" />
    </_config:LoadItem>
  </_config:LoadOrder>
</_config:DataLoadConfiguration>