Overview of the Data Load utility
The Data Load utility is an enhanced business object based loading utility. This utility provides an efficient solution for loading information into your WebSphere Commerce database. You can also customize the Data Load utility to load other types of data. The Data Load utility is the recommended loading utility.
The Data Load utility supports loading data into a workspace. When you loading data into a workspace you can make and preview changes to managed assets, without affecting what is running on your site.
- Business user
- Responsible for managing the business data.
- Developer
- Responsible for defining the data source template, business object mappings, and customizing the Data Load utility.
- Site administrator
- Responsible for the day-to-day operation of the Data Load utility.
Data Load utility - user interaction diagram
The following diagram describes how the user roles interact with the data load utility:
- The business user provides the developer with the business data.
- The developer creates a data source template, which defines how source data must be formatted before the data is loaded.
- The developer also creates the business object configuration file. The business object configuration file defines how the Data Load utility maps the input data to the business object and how to transform the business object to physical data.
- The site administrator uses the business object configuration file to define and create the load order configuration file.
- The site administrator sets the store and database settings in the environment configuration file.
- The business data is formatted according to the rules of the data source template before the data is loaded to the database.
- The formatted source data is provided to the site administrator.
- The site administrator runs the Data Load utility along with the three configuration files (environment, load order, and business object configuration files) to load the formatted source data into the WebSphere Commerce database. After the utility runs, the site administrator also verifies the results of the load.
- The business data is available in WebSphere Commerce to be managed by the business user.
For more information about how the Data Load process is structured, see Data Load utility architectural overview. For more information about how the Data Load utility works and what components are included in the process, see Data Load utility framework process and components.
Running the Data Load utility
To use the Data Load utility to load data into your WebSphere Commerce database, you must first configure or create the required files. To run the Data Load utility a business object configuration file, load order configuration file, environment configuration file, and data source file are all required. For more information, see Configuring and running the Data Load utility.
Best practices
Data Load utility limitations
Differences in the Data Load utility from WebSphere Commerce version 7
In WebSphere Commerce version 8, you can create CSV input files that include records that span multiple lines. In version 7, each record within a CSV file must be on a single line within the file. For more information, see Creating data in CSV format.
The Data Load utility in version 8 also supports loading data into columns that have a data type of NVARCHAR2.
The Data Load utility accepts an environment configuration file to be specified in the command line. When specified, it takes precedence over the element that exists in the data load environment configuration file.
Procedures and samples
Role | Topic | Summary |
---|---|---|
Site Administrator | File format for Data Load input files | A data load input file contains the actual information that the Data Load utility populates into your database. Learn about how to construct such files to ensure that the loading process is successful. |
Creating data in XML format | Learn how to create a data load input file in the supported XML format. | |
Creating data in CSV format | Learn how to create a data load input file in the supported CSV format | |
Configuring the Data Load utility to run a file difference preprocess | If you routinely load the same generated Data Load input file from an external system or source, you can choose to run a file difference preprocess. You can run this preprocess as part of the Data Load process to ensure that you are loading only new changes when you load your newest input file. | |
Configuring the CSV data reader | The CVS data reader is already provided with the Data Load utility. Learn how to configure the provided CSV data reader to change the way data is read from your CSV input files. | |
Configuring the XML data reader | An XML data reader is provided by default with the Data Load utility. Learn how to configure this provided data reader to change the way data is read from your XML formatted input files. | |
Configuring the component business object builder | The BaseBusinessObjectBuilder and
TableObjectBuilder business object builders by defining sub elements and attribute
values in the business object configuration file. |
|
Configuring the business object configuration file | Learn how to configure the business object configuration file that defines how to load the data into the database. In this file, you specify the implementation classes for your Data Reader, Business Object Builder, and Business Object Mediator components. | |
Configuring the data load order | The data load order configuration file controls the load order of the Data Load utility. Learn how to configure your data load order file. | |
Configuring a column exclusion list | You can configure a column exclusion list that causes the Data Load utility to avoid loading data into the specified columns of a table. | |
Configuring the data load environment settings | Learn how to configure the environment variables that are used by the Data
Load utility in your environment settings file. To configure the Data Load utility to load data
into a workspace, you must add the workspace attribute to the business context in the environment
settings file. Note: When you load data into a workspace, the Data Load utility respects the locking
policy set in the workspace. For more information about workspace locking policies, see Workspaces locking policies. |
|
Data Load utility command syntax | Learn how to run the utility command that runs the Data Load process | |
Verifying the results of the data load | Learn how to verify that a load operation with the Data Load utility is successfully completed. | |
Loading data into workspaces using the Data Load utility | Load data into workspaces with the data load utility | |
Loading values for single and multiple value attributes | Load data for attribute values for attributes with a single value and attributes with multiple values | |
Reuse attribute assigned values with the Data Load utility | Configure the utility to reuse assigned values for attributes when the same value is needed for multiple catalog entries. | |
Loading promotions with the Data Load utility | Load data to create or change promotions for a store. | |
Loading customer segment members by email address with the Data Load utility | Load a list of email addresses to create a customer segment and populate the segment with members that are associated with the email address. | |
Loading member group members by email address with the Data Load utility | Load members into a member group based on the member email addresses. | |
Configuring a Data Load utility scheduler job | Use the WebSphere Commerce Administration Console to configure the Data Load utility to run as a scheduled job. | |
Loading marketing objects with the Data Load utility | Load data to create marketing activities, campaigns, customer segments, content, attachments, and e-Marketing Spots. | |
Samples | Scenario: Initial load | The initial load is the scenario when you finish creating and configuring a new WebSphere Commerce instance; you then load your initial data into the WebSphere Commerce database. |
Scenario: Delta load | The delta load scenario is when your WebSphere Commerce server is up and running, then you want to insert, update, or delete your catalog, inventory, or price data. | |
Scenario: Catalog entry update load | The utility can run in an update mode to update catalog entry data. The update mode replaces or adds data for only the columns that are specified in the input file. All other columns remain unchanged. | |
Scenarios: Workspace locking for the Data Load utility | These scenarios detail the workspace locking policies that are in effect when the Data Load utility loads data into a workspace. These locking policies affect the loading of data into the workspace database. | |
../refs/rmlsampleprepare.html | This sample demonstrates how to set up the Data Load utility for its first use. | |
Catalog samples | These samples use CSV or XML files to demonstrate how to run the Data Load utility to load catalog data | |
Inventory samples | These samples use CSV files to demonstrate how to run the Data Load utility to load inventory data. | |
Member samples | These samples use CSV or XML files to demonstrate how to run the Data Load
utility to load member data. These samples include a sample for loading customer segment member data. |
|
Price samples | These samples use CSV files to demonstrate how to run the Data Load utility to load price data. | |
Location samples | These samples use CSV files to demonstrate how to run the Data Load utility to load location data. | |
Store configuration samples | This sample uses a CSV or XML file to demonstrate how to load configuration name-value pair properties for a store. | |
Commerce Composer samples | These samples use CSV file to demonstrate how to load Commerce Composer assets with the Data Load utility. | |
Promotion samples | These samples use input files to demonstrate how to load promotions, promotion folders, and load promotions into the promotion folders. | |
Sample: Loading marketing data | These samples use input files to demonstrate how to load marketing activities, campaigns, customer segments, content, attachments, and e-Marketing Spots. | |
Examples | Examples: Mapping catalog data | These examples use a CSV file to demonstrate how to insert, replace, or delete catalog data. |
Examples: Mapping inventory data | These examples use a CSV file to demonstrate how to insert, replace, or delete inventory data. | |
Examples: Mapping pricing data | These examples use a CSV file to demonstrate how to insert, replace, or delete price data. | |
Examples: Mapping catalog filter data | These examples use a CSV file to demonstrate how to insert, replace, or delete catalog filter data. | |
Examples: Mapping member data | These examples use a CSV file to demonstrate how to insert, replace, or delete member data. | |
Developer | Data Load utility architectural overview | An understanding of how the Data Load utility works, and the components that make up the Data Load utility. |
Customizing the Data Load utility | Learn how to customize elements of the Data Load utility to create custom data
readers, column handlers, business object mediators, and load extension tables. By customizing the
Data Load utility you can complete the following tasks:
|
|
Configuring an SFTP transport to retrieve external files for the Data Load utility | Customize an SFTP transport for a Data Load utility scheduled job to use to retrieve input files from an external source to load into WebSphere Commerce. |