Running tests by using Ant

When you want to automate repetitive tasks, you can run performance test assets from the command-line interface by using Ant.

Before you begin

You must have completed the following tasks:

  • Installed Test Performance.

    If you installed Test Performance on the Mac operating system, you must have added TEST_WORKBENCH_HOME as an environment variable. You must have pointed that environment variable to the installation directory of Test Performance.

    Note: On Windows® and Linux™ operating systems, the TEST_WORKBENCH_HOME environment variable is set after the installation of Test Performance is complete.

  • Created test assets within Test Performance. See Test Author Guide.

  • Installed Apache Ant. For more information, refer to the Installing Apache Ant section in the Apache Ant™ User Manual.

  • Downloaded the Test Performance Ant plugin from the HCL® License & Delivery portal onto the computer where you install Test Performance.

    For more information about specific versions of plugin, see Integration plugin compatibility matrix.

  • Closed Test Performance.

About this task

You must provide the values for certain parameters in the ANT test script to run tests from the command-line interface. The minimum parameters that you need to provide are name, workspace, projectname, and suite or configfile. The remaining parameters are optional.

For example, if you provide a complete path of a configuration file that contains the parameters for a test or schedule run in the configfile parameter, then you need not be required to provide values for the name, workspace, projectname, and suite parameters.

Note: If Test Performance is not installed in default location, then you must provide the value for the imsharedloc parameter along with other mandatory parameters.
Remember: You must consider the following requirements:

  • Enter the parameter values within the double quotation marks.

  • Ensure that the special characters in the parameter values do not break the validation of the XML file. For example, you must enter the & character as &.

Procedure

  1. Extract the following files from the downloaded Ant plugin:

    The Ant plugin includes the following files:

    • DTPerf-Ant-HCL-x.y.jar

      Where x.y is the version number of the Ant plugin.

    • ExecutePerformanceTest.xml
    • README.txt
  2. Open the ExecutePerformanceTest.xml file in a text editor and provide the required parameter values within <pt> tag by referring to the following table:

    The following table lists the required parameters that you must provide to run the test from Ant:

    Parameters Description
    configfile

    Enter the complete path to a configuration file that contains the parameters for a test or schedule run.

    For example, 
    <pt configfile=C:\Users\HCL\devopstest\workspace\Project1\config.txt />

    For more information about creating a command line config file from Test Performance, see the Related information section.
    name Enter a name for the Ant task.
    For example,
    <pt name="runtest_ant" />
    projectname

    Enter the name of the project that has performance test assets.

    For example,
    <pt name="runtest_ant" workspace="C:\Users\HCL\devopstest\workspace1" projectname="proj123"/>
    suite

    Enter the name of the test asset that you want to run.

    For example,
    <pt name="runtest_ant" workspace="C:\Users\HCL\devopstest\workspace1" projectname="proj123" 
suite="test1.testsuite"/>
    workspace

    Enter the complete path of the Eclipse workspace.

    For example,
    <pt name="runtest_ant" workspace="C:\Users\HCL\devopstest\workspace1"/>

    The following table lists the optional parameters that you can provide to run the test from Ant:

    Parameters Description
    duration

    Use this parameter to change the duration of the stages in the rate schedule.

    For example, 
    <pt name="runtest_ant" workspace="C:\Users\HCL\hclonetest\workspace1" projectname="proj123" 
suite="test1.testsuite" duration="Stage1=10s, Stage2=3m"/>

    The stage number specified must exist in the rate schedule.

    Note: The duration parameter creates a new copy of the rate schedule that contains the specified number of duration.
    exportlog

    Use this parameter to specify the path of the file directory to store the exported HTTP test log in a .txt format.

    For example,
    <pt name="runtest_ant" workspace="C:\Users\HCL\devopstest\workspace1" projectname="proj123" 
suite="test1.testsuite" exportlog="C:/Users/Documents/ANT/LogExp.txt"/>
    Note: When you do not use the exportlog parameter, then the Ant execution output is logged into the logfile.txt file and stored in a temp directory where you extracted the files of the Ant plugin.
    exportstatreportlist

    Use this parameter to specify IDs of reports that you want to export in place of the default reports.

    You can provide multiple report IDs separated by a comma. You can navigate to Preferences of Test Performance (Window > Preferences > Test > Performance Test Reports > Export Reports), and then select Show Report IDs checkbox to view the report IDs.

    You must use the exportstatreportlist parameter along with the exportstats or exportstatshtml parameter.

    For example, you can provide the value for the exportstatreportlist parameter as follows to export the Performance Report:
    <pt name="runtest_ant" workspace="C:\Users\HCL\devopstest\workspace1" projectname="proj123" 
suite="test1.testsuite" exportstats="C:/Users/Documents/ANT" exportstatreportlist="http"/>
    exportstats

    Use this parameter to provide the complete path to a directory that you can use to store the exported report in a comma-separated values (CSV) format.

    For example, you can provide the value for the exportstatreportlist parameter as follows to export the Performance Report:
    <pt name="runtest_ant" workspace="C:\Users\HCL\devopstest\workspace1" projectname="proj123" 
suite="test1.testsuite" exportstats="C:/Users/Documents/ANT"/>
    Note: If you do not specify a value for the exportstatreportlist parameter, then the reports that you selected in Preferences of Test Performance (Window > Preferences > Test > Performance Test Reports > Export Reports) are exported.
    exportstatsformat

    Use this parameter to specify a format for the result that you want to export along with the exportstats parameter.

    You must use at least one of the following options with the exportstatsformat parameter:
    • simple.csv
    • full.csv
    • simple.json
    • full.json
    • csv
    • json
    For example, 
    <pt name="runtest_ant" workspace="C:\Users\HCL\devopstest\workspace1" projectname="proj123" 
suite="test1.testsuite" exportstats="C:/Users/Documents/ANT" exportstatsformat="simple.json"/>

    You can add multiple options separated by a comma.

    For example, 
    <pt name="runtest_ant" workspace="C:\Users\HCL\devopstest\workspace1" projectname="proj123" 
suite="test1.testsuite" exportstats="C:/Users/Documents/ANT" exportstatsformat="simple.json,full.csv"/>

    When you want to export both simple and full type of test results in a json or csv format, you can specify either json or csv as options.

    For example, 
    <pt name="runtest_ant" workspace="C:\Users\HCL\devopstest\workspace1" projectname="proj123" 
suite="test1.testsuite" exportstats="C:/Users/Documents/ANT" exportstatsformat="json"/>
    When the test run completes, the test result exports to simple.json and full.json files.
    Note: The values provided in the ExecutePerformanceTest.xml file always take precedence over the Export Reports options set in the Preferences of Test Performance (Window > Preferences > Test > Performance Test Reports > Export Reports).
    exportstatshtml

    Use this parameter to provide the complete path to a directory that you can use to export web analytic results.

    The results are exported to the specified directory. You can analyze the results on a web browser without using Test Performance.

    For example,
    <pt name="runtest_ant" workspace="C:\Users\HCL\devopstest\workspace1" projectname="proj123" 
suite="test1.testsuite" exportstatshtml="C:/Users/Documents/ANT"/>
    Note: When you do not use the exportstatshtml parameter, then the Performance report is stored in a temp directory where you extracted the files of the Ant plugin.
    history

    Use this parameter when you want to view a record of all events that occurred during a test or schedule run.

    You can use any of the following options:
    • jaeger: To send test logs to the Jaeger UI during the test or schedule run.
      Note: You must set the JAEGER_AGENT_HOST property as an environment variable by using the command line before you use the jaeger option to send test logs to the Jaeger UI.

    • testlog: To send test logs as traditional test logs in Test Performance during the test or schedule run.

    • null: To send no test logs either to the Jaeger UI or Test Performance during the test or schedule run.

    For example,
    <pt name="runtest_ant" workspace="C:\Users\HCL\devopstest\workspace1" projectname="proj123" 
suite="test1.testsuite" history="jaeger"/>

    You can add multiple options separated by a comma to send test logs during the test or schedule run to Test Performance and the Jaeger UI.

    For example,
    <pt name="runtest_ant" workspace="C:\Users\HCL\devopstest\workspace1" projectname="proj123" 
suite="test1.testsuite" history="jaeger,testlog"/>

    For more information about how to view test logs in the Jaeger UI and Test Performance, see the Related information section.
    imsharedloc

    Use this parameter to enter the complete path to the HCLIMShared location, if it is not in the default location.

    The default location of the IMShared directory is as follows:
    Operating system The default path to the directory
    Windows® C:\Program Files\HCL\HCLIMShared
    Linux™ /opt/HCL/HCLIMShared
    Mac /Application/HCL/HCLIMShared
    For example, 
    <pt name="runtest_ant" workspace="C:\Users\HCL\devopstest\workspace1" projectname="proj123" 
suite="test1.testsuite" imsharedloc="D:\Testtool\HCL\devopstest"/>
    labels

    Use this parameter to add labels to test results when the test run is complete.

    For example, 
    <pt name="runtest_ant" workspace="C:\Users\HCL\devopstest\workspace1" projectname="proj123" 
suite="test1.testsuite" labels="label1, label2"/>

    You can add multiple labels to a test result separated by a comma. When you run test assets, then the labels that you added are displayed on the Performance Report in Test Performance.

    The Results page of Test Hub displays the label that you added, in the specific test asset in the following conditions:

    • When you set Publish result after execution as Always or Prompt in the Preferences of Test Performance (WindowsPreferencesTestDevOps Test Hub).

    • When you use the labels parameter along with the publish parameter.

    Note: If the value of the labels parameter contains double quotation marks (""), for example, "100" users, then the test run fails due to non-validation of xml file.

    To work around this problem, you must create a command-line config file, and then run the test by using the configfile parameter.
    overridermlabels
    Use this parameter to perform any of the following actions:

    • Enable the Resource Monitoring from Service option for a performance schedule if the Resource Monitoring from Service option is not enabled from the schedule editor in Test Performance.

    • Ignore Resource Monitoring sources that were set in the performance schedule and change for a label matching mode.

    • Replace an existing set of Resource Monitoring labels that were set in the performance schedule and run the schedule with a new set of Resource Monitoring labels.

    If you have added a label in Test Hub for a Resource Monitoring source as rm1, then you can provide value as shown in the following example to collect data from the source:

    <pt name="runtest_ant" workspace="C:\Users\HCL\devopstest\workspace1" projectname="proj123" 
suite="test1.testsuite" overridermlabels="rm1"/>

    If your label for resource monitoring contains a comma (,) then you must replace the single comma with the double comma while providing the value. For example, if you added a label to a Resource Monitoring source as rm1,test, then you must provide the value as shown in the following example to collect data from source:

    <pt name="runtest_ant" workspace="C:\Users\HCL\devopstest\workspace1" projectname="proj123" 
suite="test1.testsuite" overridermlabels="rm1,,test"/>
    Notes:

    • You can use this field only when you want to run a Rate schedule or VU schedule.

    • You can add multiple Resource Monitoring labels separated by a comma.

    • You must add the Resource Monitoring labels to the Resource Monitoring sources on the Resource Monitoring page in the Test Hub project.
    overwrite Set this parameter value to true or false to determine whether a result file with the same name must be overwritten or not.

    The default value is false.

    For example,
    <pt name="runtest_ant" workspace="C:\Users\HCL\devopstest\workspace1" projectname="proj123" 
suite="test1.testsuite" overwrite="true"/>
    publish

    Use this parameter to publish test results to Test Hub by using ANT.

    Remember: Before you use the publish parameter, you must provide the offline user token of Test Hub by using any of the following methods:

    • Set the value of the HCL_DEVOPS_OFFLINE_TOKEN environment variable to the offline user token of Test Hub by using the command-line interface.

    • Provide the offline user token of Test Hub in the preferences of Test Performance (Window > Preferences > Test > DevOps Test Hub)

    You must use one of the following values in the publish parameter:

    • serverURL#project.name=name_of_the_project&teamspace.name=name_of_the _teamspace

    • serverURL#project.name=name_of_the_project&teamspace.alias=name_of_the _teamspace_alias

    • no

      The no option is useful if the product preferences are set to publish the results, but you do not want to publish them.

    For example: 
    <pt name="runtest_ant" workspace="C:\Users\HCL\devopstest\workspace1" projectname="proj123" 
suite="test1.testsuite" publish= "URL_OF_DevOpsTest_Hub#project.name=test&amp;teamspace.name=ts1"/>
    <pt name="runtest_ant" workspace="C:\Users\HCL\devopstest\workspace1" projectname="proj123" 
suite="test1.testsuite" publish=no"/>
    Where:

    • URL_OF_Test Hub is the URL of Test Hub.

    • test is the name of the project in Test Hub.

    • ts1 is the name of the team space.

    Notes:

    • If you have a project with the same name in different team spaces, then you must append either the &teamspace.name=name_of_the _teamspace or &teamspace.alias=name_of_the_teamspace_alias options.

    • If the name of the project or team space contains a special character, then you must replace it with %<Hexvalue_of_special_character>.

      For example, if the name of the team space is Initial Team Space, then you must provide it as Intial%20Team%20Space.

      Where %20 is the Hex value of Space character.

    • The values provided in the Publish field always take precedence over the Results options set in the product preferences (Window > Preferences > Test > DevOps Test Hub > Results).

    The Reports information section on the Log file displays the names of the report along with its corresponding URLs in the following conditions:
    • When you configured the URL of Test Hub in Preferences of Test Performance (Window > Preferences > Test > DevOps Test Hub).
    • When you set Publish result after execution as Always or Prompt in the Preferences of Test Performance (Window > Preferences > Test > DevOps Test Hub > Results).
    • When you used publish parameter or publish parameter along with publishreports parameter.
    publish_for

    Use this parameter to publish test results to Test Hub by using ANT based on the completion status of the tests.

    You must use the publish_for parameter along with the publish parameter. You must add multiple parameters separated by a comma. The available options that you can use for the publish_for parameter are as follows:

    • ALL: You can use this option irrespective of status of the test.

    • PASS: You can use this option to publish test results for the tests that have passed.

    • FAIL: You can use this option to publish test results for the tests that have failed.

    • ERROR: You can use this option to publish test results for the tests that included errors.

    • INCONCLUSIVE: You can use this option to publish test results for the inconclusive tests.

    For example,
    <pt name="runtest_ant" workspace="C:\Users\HCL\devopstest\workspace1" projectname="proj123" 
suite="test1.testsuite" publish= "URL_OF_DevOpsTest_Hub#project.name=test&amp;teamspace.name=ts1" publish_for="FAIL,ERROR"/>
    publishreports

    Use this parameter to publish specific test results to Test Hub.

    You must use the publishreports parameter along with the publish parameter. The options that you can use with publishreports are as follows:

    • STATS

    • TESTLOG

    For example, 
    <pt name="runtest_ant" workspace="C:\Users\HCL\devopstest\workspace1" projectname="proj123" 
suite="test1.testsuite" publish= "URL_OF_DevOpsTest_Hub#project.name=test&amp;teamspace.name=ts1"
publishreports="STATS"/>

    You can prefix the publishreports value with ! to publish the reports except for the specified one in the test script.

    For example, 
    <pt name="runtest_ant" workspace="C:\Users\HCL\devopstest\workspace1" projectname="proj123" 
suite="test1.testsuite" publish= "URL_OF_DevOpsTest_Hub#project.name=test&amp;teamspace.name=ts1"
publishreports="!TESTLOG"/>
    rate

    Use this parameter to change the rate of the rate runner group.

    For example, 
    <pt name="runtest_ant" workspace="C:\Users\HCL\devopstest\workspace1" projectname="proj123" 
suite="test1.testsuite" rate="Rate Runner Group1=1/s, 3/m"/>

    Where, Rate Runner Group1 is the name of the rate runner group that has two stages. The desired rate for the first stage is one iteration per second and the rate for the second stage is three iterations per minute.

    Notes:
    • The name of the Rate Runner group must match with the name in the rate schedule.
    • The rate parameter creates a new copy of the rate schedule that contains the specified number of rate.
    results

    Use this parameter to provide a different name to the result file.

    The default name of the result file is the name of the test or schedule with a timestamp appended.
    swapdatasets

    Use this parameter to replace the dataset values during a test or schedule run.

    You must ensure that both original and new datasets are in the same workspace and have the same column names.When you enter a value for the swapdatasets parameter, you must also include the path to the dataset. You must provide the values for the swapdatasets parameter in the following format:

    /project_name/ds_path/original_ds.csv:/project_name/ds_path/new_ds.csv

    For example,
    <pt name="runtest_ant" workspace="C:\Users\HCL\devopstest\workspace1" projectname="proj123" 
suite="test1.testsuite" swapdatasets="/proj1023/Datasets/ds1.csv:/proj1023/Datasets/new_ds1.csv"/>

    You can override multiple datasets that are saved in a different project by adding multiple paths to the dataset separated by a semicolon. For a test or schedule, the default value is the dataset specified in the test editor or schedule editor.
    usercomments

    Use this parameter to add text that you want to display in the user comments row of the report.

    For example,
    <pt name="runtest_ant" workspace="C:\Users\HCL\hclonetest\workspace1" projectname="proj123" 
suite="test1.testsuite" usercomments="test run with dataset"/>
    Note: If the value of the usercomments parameter contains double quotation marks (""), for example, test run with "dataset", then the test run fails due to non-validation of xml file.

    To work around this problem, you must create a command-line config file, and then run the test by using the configfile parameter.
    users

    Use this parameter to overrides the default number of users, if required.

    For a schedule, the default is the number of users specified in the schedule editor and for a test, the default is one user.

    Note: The users parameter creates a new copy of the schedule that contains the specified number of users.
    varfile Use this parameter to provide a complete path to an XML file that contains the variable name and value pairs.
    vmargs

    Use this parameter to specify the maximum heap size for the Java process.

    You can add multiple VM Arguments separated by a comma.
    Note: You can add multiple <pt> tasks and provide the details for each test to run multiple tests simultaneously.
  3. Save the DTPerf-Ant-HCL.xml file.
  4. Open the command prompt, and then change the directory to a location where you downloaded the Ant plugin.
    For example, 
    cd D:\Downloads\Ant\DTPerf-Ant-HCL-x.y

    Where x.y is the version number of the Ant plugin.

  5. Run the following command to initiate the test run: 
    ant -f DTPerf-Ant-HCL.xml

Results

You have run the test from the command-line interface by using Ant.

What to do next

You can view the Ant execution output from the logfile.txt file that is stored in a temp directory. The temp directory is located where you extracted the files of the Ant plugin. If you provided the location for the exportlog parameter, then the Ant execution output is stored in the specified location.

The Reports information section on the Log file displays the names of the report along with its corresponding URLs if you publish test results to Test Hub.