Data Extract utility

The Data Extract utility, which uses the Data Load utility framework, extracts data from the HCL Commerce database into an output file.

You can run this utility in the staging and production environments. However, you are recommended to run the utility in an environment that has all of the information for that you need to extract for an object. For example, the staging environment might not have inventory or pricing information for a catalog entry. In this case, run the utility on the production environment.

For general information about the Data Extract utility, see Overview of the Data Extract utility.

Note: If you integrate HCL Commerce with IBM Product Recommendations, use this utility to generate Enterprise Product Report (EPR) data for use with . Before you extract this data, you must configure the utility to generate ECDF and EPCMF files in the correct format to load into IBM Product Recommendations. For more information, see Data extraction utility for dynamic recommendations in.

Command syntax


Data Extract utility syntax diagram

Parameter values

full_path
Is the full path location of the order configuration file for the Data Extract utility.
-DXmlValidation
(Boolean) Used to turn on or off XML validation. For accuracy, the defined configurations inside the XML files are validated against the Data Extract utility and Data Load utility XSDs. By default, XML validation is turned on (true). If you want to turn off validation, enter -DXmlValidation=false.
-DLoadOrder
Specifies the order that data is extracted from the database. This parameter overrides the LoadOrder element in the order configuration file. You can specify any number of object items to extract that you want, with each item separated by a comma. Ensure that parameter values do not contain white space. If there is white space, enclose the parameters in double quotation marks. For example, 
"-DLoadOrder=loadItemName1, loadItemName2"
-DConsoleHandler.level
Used to override the console logging level that is defined in the utilities_root\dataextract\logging.properties file. For example, enter -DConsoleHandler.level=SEVERE to change the console logging level to SEVERE.
-DFileHandler.level
Used to override the file logging level that is defined in the utilities_root\dataextract\logging.properties file. For example, enter -DFileHandler.level=WARNING to change the file logging level to WARNING.
-Dpackage_name.level
Where package_name is the name of any package. This parameter is used to override the log level for the specified package. If the package_name is blank (-D.level=level name), then the log level is changed for all packages. The level name can be: OFF, SEVERE, WARNING, INFO, CONFIG, FINE, FINER, FINEST, or ALL.
To help troubleshoot errors that can occur when you run the Data Extract utility, you can override logging levels. For example, override the following Data Load utility packages that the have logging available that the Data Extract utility can use:
-Dcom.ibm.commerce.foundation.dataload.businessobjectbuilder.level
-Dcom.ibm.commerce.foundation.dataload.businessobjectmediator.level
-Dcom.ibm.commerce.foundation.dataload.config.level
-Dcom.ibm.commerce.foundation.dataload.datareader.level
-Dcom.ibm.commerce.foundation.dataload.datawriter.level
-Dcom.ibm.commerce.foundation.dataload.idresolve.level
-Dcom.ibm.commerce.foundation.dataload.database.level
-Dcom.ibm.commerce.foundation.dataload.level
For more information about changing the logging level for a package, see Configuring logging for runtime environments. For more information about logging, see Logging services.

Running the Data Extract utility

  1. HCL Commerce DeveloperOn a command line, go to the WCDE_installdir\bin directory.
  2. LinuxOpen a command line in the . Change the directory to utilities_root/bin directory. For information about entering and leaving containers, see Running utilities from the Utility server Docker container.
  3. Run the following command to use Data Extract utility to run the specified order configuration file. The file identifies the business object configuration files that define how to extract the data.
    • Linux./dataextract.sh ../path_to_order_configuration_file/wc-dataextract.xml
    • HCL Commerce Developerdataextract ..\path_to_order_configuration_file\wc-dataextract.xml
    Where
    path_to_order_configuration_file
    The relative path to the order configuration file that identifies the configuration files that you are using with the Data Extract utility.
    wc-dataextract.xml
    The order configuration file. Sample files are provided with HCL Commerce in the following directories:
    • Linuxutilities_root/samples/DataExtract
    • HCL Commerce DeveloperWCDE_installdir\samples\DataExtract
    By default the sample order configuration file is typically named wc-dataextract.xml or wc-dataextract-object.xml, where object is the type of object that is being extracted. These files are structured similar to the order configuration files that are used to configure the Data Load utility, but they include some extensions.
    As an example, the following command runs the Data Extract utility to extract promotion objects
    • Linux./dataextract.sh ../samples/DataExtract/Promotion/wc-dataextract.xml
    • HCL Commerce Developerdataextract ..\samples\DataExtract\Promotion\wc-dataextract.xml
Note: Since the Data Extract utility uses the Data Load utility framework, you can also configure the Data Load utility to extract data. You can configure the Data Load utility command to use some of the Data Extract utility configuration files in the subfolders of the following directory:
  • Linuxutilities_root/samples/DataExtract
  • HCL Commerce DeveloperWCDE_installdir\samples\DataExtract
You cannot configure the Data Load utility to extract catalog data for generating Enterprise Product Report (EPR) data for use with IBM Product Recommendations. You must use the Data Extract utility to extract this data.

Troubleshooting

You can encounter errors when you run the utility for many reasons. Errors can be caused by incorrect command parameters, configurations, and more. To help troubleshoot errors that are encountered, review the wc-dataextract.log file as the first step in determining the source of your error. This file can be located within the following directory

  • Linuxutilities_root\logs
  • WCDE_installdir\logs
By default, this log file provides only a summary report. You can however specify the level of package logging when you run the utility to generate more information about the extract process within this log file. For example, to run the utility with a finer log level for catalog information, you can specify the following command that includes the -Dpackage_name.level parameter:
dataextract.sh wc-dataextract.xml -Dcom.ibm.commerce.catalog.dataload.level=FINER