Running tests from the command line

You can run test scenarios individually or all at once by using an Ant script. The script generates a report that contains the results of the tests.

Before you begin

  • The store that you want to test is set up.
  • Install Java JDK 1.8 or higher, and ensured that the system environment variable JAVA_HOME points to the JDK installation directory.
  • Install Apache Ant 1.8.1 or higher. Ensure that the ANT_HOME environment variable points to your Ant installation directory, and ANT_HOME/bin is in your system path.
  • Extract the StorefrontTestAssets.zip package to a directory of your choice, if you do not intend to use the same files as your Eclipse-based environment installation.
  • Installing the Storefront Test Automation Engine and its dependencies; if you have an Eclipse-based environment that is installed, you can follow these instructions. If you do not have an Eclipse-based environment that is installed, you must manually extract the StorefrontTestAssets.zip projects into a directory of your choice. Copy the WebSphere Commerce service layer JAR files and Eclipse EMF-related JAR files to the lib directory of the StarterStoreTestAssetsDependencies project that is described in the topic by using basic operating system copy commands instead of through Eclipse.
  • Importing a test bucket project.
  • Changed the following properties in the config.properties file to reflect your environment. This file stores the configuration properties for running tests and is in the root of the test bucket project for the store.
    HOSTNAME
    The host name of the store web server.
    STOREURL
    The URL to the root of the store you are testing.
    BROWSER
    The web browser that you are using for your testing. For example, FIREFOX or INTERNET_EXPLORER.
    ADMIN_USER_NAME
    The site administrator user name.
    ALLOW_MODIFY_CHANGE_FLOWS
    Determines whether the change flow options can be changed in a test (true) or not (false).
    PRODUCT_TYPE
    Valid values are TOOLKIT and RUNTIME, with RUNTIME as the default. Identifies if the test is being run against a WebSphere Commerce Toolkit or Runtime server.

About this task

You can run tests by using the following modes:
Batch test execution
Since the results of each test execution are archived, you can easily run the same script multiple times with different test cases specified in the build.properties file.
Custom test bucket execution
To have your own custom test buckets that are run from the command line in a similar fashion, copy the Ant subdirectory from a test bucket to your own test bucket. If your test bucket is dependent only on the WCWebTestEnginge, copy the Ant directory that is found in Aurora-Tests-Wte. Edit the build.xml file and modify the project name attribute to represent the project name of your test bucket.

Procedure

  1. Go to the directory of the test bucket project that you want to run.
    For example, for the Aurora starter store, enter: Aurora-Tests.
  2. Ensure that you modify the config.properties file to set the properties to match your environment.
  3. Go to the Ant subdirectory.
    For example, enter: Aurora-Tests/ant.
  4. If you do not want to run all of the tests, edit the build.properties file to change the value of the test.include property to the specific test that you want to run.
    For example: test.include=**/FSTOREB2C_00.class.The build.properties file contains several other properties that detail the location of the output, Java build parameters, and test parameters. Leave these properties at their default settings, however, if you want to make changes, comments are provided to help you determine customized values.
  5. In the command line, change the directory to your Ant directory.
    For example, workspace_dir\Aurora-Tests-Wte\ant.
  6. Type ant to run the tests. The build.xml is configured to run the specified tests as the default action.
    The test runs; the specified web browser opens, closes, and moves from page to page according to the test. After the test completes, the Ant script stops and you are returned to the command prompt.

Results

The results of a set of tests are stored in the test archive folder. By default, this folder is in the target/doc/archive subdirectory of the project. This folder contains one or more timestamped directories that represent the different test runs.

The test report includes a Packages section that displays the name of the test bucket and a Classes section that lists the test scenarios that make up the test bucket. Four test characteristics are also summarized in the report.
Failures and Errors
Cause the test to fail. Clicking the number links to a page that displays the test scenario and the corresponding test bucket that caused the failure or error. Details are also provided to help correct the issue.
Success rate
The percentage of automation scripts that passed.
Total run time
The total amount of time (in seconds) for all the automation scripts to run.

The detailed report that is generated in batch mode simplifies the process of locating and correcting failures in your automation scripts. For each test scenario, the report lists the four main characteristics, a time stamp of when it completed, and the host name of the virtual machine that ran it.