Overview of the Data Load utility

Data Load utility - user interaction diagram

The following diagram describes how the user roles interact with the data load utility:

1. The business user provides the developer with the business data.
2. The developer creates a data source template, which defines how source data must be formatted before the data is loaded.
3. The developer also creates the business object configuration file. The business object configuration file defines how the Data Load utility maps the input data to the business object and how to transform the business object to physical data.
4. The site administrator uses the business object configuration file to define and create the load order configuration file.
5. The site administrator sets the store and database settings in the environment configuration file.
6. The business data is formatted according to the rules of the data source template before the data is loaded to the database.
7. The formatted source data is provided to the site administrator.
8. The site administrator runs the Data Load utility along with the three configuration files (environment, load order, and business object configuration files) to load the formatted source data into the WebSphere Commerce database. After the utility runs, the site administrator also verifies the results of the load.
9. The business data is available in WebSphere Commerce to be managed by the business user.

For more information about how the Data Load process is structured, see Data Load utility architectural overview. For more information about how the Data Load utility works and what components are included in the process, see Data Load utility framework process and components.

Running the Data Load utility

To use the Data Load utility to load data into your WebSphere Commerce database, you must first configure or create the required files. To run the Data Load utility a business object configuration file, load order configuration file, environment configuration file, and data source file are all required. For more information, see Configuring and running the Data Load utility.

Best practices

Data Load utility limitations

• Newly created business objects might not show up immediately in the storefront due to caching. To resolve this problem, invalidate the dynamic cache manually. For more information about cache invalidation, see Removing cache entries through the Cache Monitor.
• When you update business objects, the changes might not show up in the Management Center. This issue can occur due to caching. To resolve this problem, invalidate the data cache. For more information, see Troubleshooting: Business object changes are not reflected in the Management Center.
• You can load data in only CSV and XML formatted files. By default, most sample configuration files that are provided with WebSphere Commerce are configured to load CSV files. To load XML files, you must configure the utility to use the XML data reader. Your XML input files must also use a CSV-like structure. For more information, see File format for Data Load input files.

  If you want to load data in other formats, or in an XML structure that is not supported by the default XML data reader, you must create your own custom data reader.

• You must configure the Data Load utility to use a business object mediator to map your input file data with the appropriate WebSphere Commerce business objects. By default, you are provided with business object mediators for loading data for business objects in the following components:
  • Catalog
  • Inventory
  • Price and Catalog filter
  • Member
  • Location
  • Commerce Composer
  • Promotions
  • Marketing
  To load other data, you can use the com.ibm.commerce.foundation.dataload.businessobjectmediator.TableObjectMediator, or you can create your own custom business object mediators.
• The TableObjectMediator does not support workspace locking.
• If your site uses WebSphere Commerce search, the delta search index might not rebuild correctly when you delete some catalog objects with the Data Load utility in delete mode. When you delete a child object of a catalog entry or category with the utility in delete mode, both the child and parent object are removed from the delta search index rebuild. This removal can cause the parent catalog entry or category to no longer be indexed or display correctly in the storefront.

  Use the utility in replace mode to delete catalog objects when your site uses WebSphere Commerce search. To delete objects with the utility in replace mode, include the value 1 for the Delete column of an object in your input file. If you do decide to delete catalog objects with the utility in delete mode, run a full index rebuild after the load operation completes.

• The member component business object mediators do not support the following actions with the Data Load utility:
  • Deleting users or organizations
  • Loading user passwords
  • Modifying the parent organization of a member
  • Modifying the distinguished name (DN) of an organization
  • Modifying the entity type of an organization
• When you are loading userData into custom extension database tables, you cannot load data into columns that have a data type that is not supported by the utility.

Differences in the Data Load utility from WebSphere Commerce version 7

In WebSphere Commerce version 8, you can create CSV input files that include records that span multiple lines. In version 7, each record within a CSV file must be on a single line within the file. For more information, see Creating data in CSV format.

The Data Load utility in version 8 also supports loading data into columns that have a data type of NVARCHAR2.

The Data Load utility accepts an environment configuration file to be specified in the command line. When specified, it takes precedence over the element that exists in the data load environment configuration file.

Procedures and samples