Properties of Steps

You must set the properties of steps to run the tests.

Input properties for the test run step

You must provide the values for the following required fields:
  • Workspace
  • Project
  • Test Suite Name
If you include these required parameters in a configuration file and use the Config File field to specify the complete file path, then these values are not required.
Field Description
Required
Name Enter the name of the step.
Workspace Enter the complete path to the Eclipse workspace.
Project Enter the path, including the file name of the project relative to workspace.
Test Suite Name Enter the path, including the file name, of the test to run relative to the project. A test can be a Web UI test, compound test, or Accelerated Functional Test. To run multiple tests sequentially, specify test names separated by comma.
Optional
IMShared Location Enter the complete path to the location of the HCLIMShared folder. For example, C:\Program Files\HCL\HCLIMShared
Import Enter the complete path to the project folder that is cloned from a source control system or remote repository. When you use this option, you can specifiy a name for the new workspace to be created to run the imported test assets in the Workspace field.
Var File Enter the complete path to the XML file that contains the variable name and value pairs.
Config File Enter the complete path to a file that contains the parameters for a test run.
Results File Enter the name of the results file. The default result file is the test name with a time stamp appended.
Overwrite Results file Determines whether a results file with the same name is overwritten. The default value is true which indicates that the results files can be overwritten.
Number of Virtual Users For a schedule, the default value is the number of users specified in the schedule editor. For a test, the default value is one user. Overrides the default number of users, if required.
VM Args Java™ virtual machine arguments to pass in.
Protocol Input Choose one of the following options to customize the test run:
  • Run a Web UI test in parallel on different browsers.

    For example, Protocol Input is all.available.targets.in.parallel=all or

    Protocol Input is all.available.targets.in.parallel=chrome,ff,ie

    Note: If you use the Protocol Input argument, you must not use the equivalent VM Args arguments:
    VM Args = -Dall.available.targets.in.parallel=all
VM Args = -Dall.available.targets.in.parallel=browser1,browser2,browser3
  • Run a UI test in the background through an agent.

    You can use bg=true to run a UI test in the background. The test execution continues even when the screen is locked or the agent is disconnected.
Dataset Override For a test, the default value is the dataset specified in the test editor.
Note:

You must use the Dataset Override option to replace the dataset values during a test run. You must ensure that both original and new datasets are in the same workspace and have the same column names. You must also include the path to the dataset.

For example,

/project_name/ds_path/ds_filename.csv:/project_name/ds_path/new_ds_filename.csv.

You can swap multiple datasets that are saved in a different project by adding multiple paths to the dataset separated by a semicolon (;).

For example, 

/project_name1/ds_path/ds_filename.csv:/project_name1/ds_path/new_ds_filename.csv;
/project_name2/ds_path/ds_filename.csv:/project_name2/ds_path/new_ds_filename.csv
Exported HTTP Test log File Enter the complete path to a file in which to store the exported HTTP test log.
Exported Statistical Report Data File Enter the complete path to the directory in which to store exported statistical report data.
Custom Report Format Files A comma-separated list of absolute paths to custom report format files (.view files) to use when exporting statistical report data with the -exportstats option.
User Comments Enter the text within double quotation mark to display it in the User Comments row of the report.
Note:
  • When you run tests by using the double quotation marks ("") for the User Comments field, then the User Comments row of a report does not contain double quotation marks.
  • To work around this problem, you must create a command-line config file, and then run the test by using the Config File field.
Export Report Specify the values to export the unified report of UI tests to the file formats such as PDF, HTML, and XML.

Enter the following details in the text field:

type=<report type>;format=<file type>;folder=<destination folder path>;filename=<name of the exported file>

For example, type=unified;format=pdf,xml,html;folder=E:\temp\logs;filename=UCDExecutionReport

When you choose to export the report to all the supported formats, the report in different file formats use the same file name that is specified in the field.
Exportstatsformat

Specify a format for the report that you want to export. You must use the Exported Statistical Report Data File parameter along with the Export stats format parameter.

You must use at least one of the following formats:

  • simple.csv

  • full.csv

  • simple.json

  • full.json

  • csv

  • json

For example, 
Exported Statistical Report Data File = <local_dir_path> 
Export stats format = json

You can add multiple formats for the report separated by a comma (,).

If you want to export both the simple and full reports in a json or csv format, you can specify json or csv as the format in the field.

The reports are saved to the location specified in the Exported Statistical Report Data File field.
Labels Enter the labels to be added to the test results when the test run is complete. You can add multiple labels to a test result separated by a comma.

For example, label1, label2

When you run test assets then the same labels are displayed on the UI Test Statistical Report in Test UI.

If you have set Publish result after execution as Always or Prompt in the Test UI preferences (Window > Preferences > Test > DevOps Test Hub) and use the Labels option, then the Results page of DevOps Test Hub displays the same label for the specific test asset.

Note:
  • When you run tests by using the double quotation marks ("") for the Labels field, then the labels in the test result do not include the double quotation marks.
  • To work around this problem, you must create a command-line config file, and then run the test by using the Config File field.
  • When you use the Config File field to run tests, then labels provided in the configuration file take precedence over the labels provided in the Labels field.
Exportstatshtml Enter the complete path to a directory to export web analytic results. The results are exported to the specified directory. You can then analyze the results on a web browser without using the test workbench.
Publish Specify the server URL and project name to publish test results to DevOps Test Hub.

You must provide the URL and offline user token of the server in Window > Preferences > Test > DevOps Test Hub of Test UI before you use the publish parameter in the test script.

Use the following arguments with the Publish parameter:

  • To specify the project name, use any of the following formats:

    • serverURL #project.name=projectName&teamspace.name=name_of_the _teamspace

    • serverURL #project.name=projectName&teamspace.alias=name_of_the _teamspace_alias

    You must consider the following points while providing the project name:

    • If the project name is not specified, then the value of the Project parameter is used.

    • If you have a project with the same name in different team spaces, then you can append either the &teamspace.name=name_of_the _teamspace or &teamspace.alias==name_of_the_teamspace_alias options.

      For example: 
      Workspace = C:/Users/IBM/devopstest/workspace1
Project = proj1
Test Suite Name = Tests/testHttp.testsuite
Publish = https://localhost:5443#project.name=test&teamspace.name=ts1
      Where:

      • https://localhost:5443 is the URL of the server.

      • test is the name of the project.

      • ts1 is the name of the team space.

    • If the name of the project or team space contains a special character, then you must replace it with %<Hex_value_of_special_character>.

      For example, if the name of the team space is Initial Team Space, then you must provide it as Intial%20Team%20Space.

      Where, %20 is the hexadecimal value of the space character.

  • To avoid publishing of reports, use no.

    You can use the no option if you do not want to publish test results after the run. This option is useful if the product preferences are set to publish the results, but you do not want to publish them.

    For example:
    Workspace = C:/Users/IBM/devopstest/workspace1
Project = proj1
Test Suite Name = Tests/testHttp.testsuite
Publish = no

If you do not use the Config File parameter to run the tests, then the values provided in the test script always take precedence over the Results options set in the product preferences (Window > Preferences > Test > DevOps Test Hub > Results).

The Reports information section on the Output window displays the names of the report along with its corresponding URLs in the following conditions:

  • When you configured the URL of DevOps Test Hub in preferences of Test UI (Window > Preferences > Test > DevOps Test Hub.

  • When you set Publish result after execution as Always or Prompt in the preferences of Test UI (Window > Preferences > Test > DevOps Test Hub > Results).
Publish_for

Specify the option to publish test results to DevOps Test Hub based on the completion status of the tests. You must use the Publish_for parameter along with the Publish parameter.

The following are the available options that you can use for the Publish_for parameter:

  • ALL - This is the default option. You can use this option to publish test results for any text execution verdict.
  • PASS - You can use this option to publish test results for the tests that have passed.
  • FAIL - You can use this option to publish test results for the tests that have failed.
  • ERROR - You can use this option to publish test results for the tests that included errors.
  • INCONCLUSIVE - You can use this option to publish test results for the inconclusive tests.
You can add multiple parameters separated by a comma.
For example: 
Workspace = C:/Users/IBM/devopstest/workspace1
Project = proj1
Test Suite Name = Tests/testHttp.testsuite
Publish = https://localhost:5443#project.name=test&teamspace.name=ts1
Publish_for = FAIL,ERROR
Publishreports Specify the option to publish test results in DevOps Test Hub. The values that you can use with Publishreports are as follows:
  • FT - This is an identifier for Functional Test Report. You can use this value to publish the unified report if it is available for the selected test. See Unified reports.
  • STATS - This is an identifier for Statistics Report. You can use this value to publish the web analytics report if it is available for the selected test. See UI Test Statistical report.
  • TESTLOG - This is an identifier for Test Log. You can use this value to publish the test log if it is available for the selected test. See Logs overview.

You must use the Publishreports parameter along with the Publish parameter.

For example: 
Workspace = C:/Users/IBM/devopstest/workspace1
Project = proj1
Test Suite Name = Tests/testHttp.testsuite
Publish = https://localhost:5443#project.name=test&teamspace.name=ts1
Publishreports = STATS, TESTLOG

The values specified here override the values selected in Window > Preferences > Test > DevOps Test Hub > Results of Test UI.

You can prefix the value with ! to publish the reports except for the specified one in the test script.

For example, 
Workspace = C:/Users/IBM/devopstest/workspace1
Project = proj1
Test Suite Name = Tests/testHttp.testsuite
Publish = https://localhost:5443#project.name=test&teamspace.name=ts1
Publishreports = !TESTLOG

All the reports except the TESTLOG report is published to DevOps Test Hub after executing the command.