Running tests by using the command line

You can run tests by using the HCL DevOps Test Integrations and APIs (Test Integrations and APIs) command line.

The command has two distinct usages: By specifying all the arguments on the command line, and by using a parameter file to specify one or more configurations.

When you run a single configuration (specified by the -environment, -run, and -project arguments), you can specify all the arguments on the command line. An example follows:

RunTests   [-noHTTP] [-noResultsPublishers] [-useResultsPublishers <pub_1>,<pub_2>]
[-noPublishers] [-resultsServerLogging absolute]
[-overrideSlowFail scenario on] [-projectUser <user>] [-projectPass <password>]
[-loginAsAdmin] [-junitDir <directory>] [-securityToken <token>] [-input <file>]
[-output <file>] [-environmentTags <env>] [-environment <name>]
[-project <file>] [-run <res_1>;<res_2>]
You can also use a parameter file to specify one or more configurations. An example follows: 

RunTests   [-noHTTP] [-noResultsPublishers] [-useResultsPublishers <pub_1>,<pub_2>]
[-resultsServerLogging ignore] [-securityToken <token>]
[-parameterFile <file>]
Note: If you are specifying multiple test asset resources, the assets must be separated by semicolons and enclosed in quotation marks.
For example:

RunTests [-noHTTP] [-useResultsPublishers MyPub,TIB_Pub] [-resultsServerLogging ignore] [-environment HotelApp] [-project 
C:\Projects\HotelApp\HotelApp.ghp] [-run 
"HotelApp/Airline/booking/MakeBooking/CardType = Visa;HotelApp/Airline/booking/MakeBooking/CardType = Master"]
Notes:
  • When you use the RunTests command, if all resources that are to be run are stubs, they run in parallel. Any other combinations of resources are run serially. To correctly manage running tests in parallel to stubs, test suites must be created and run.
  • If any of the command-line parameters include spaces, they must be enclosed in quotation marks.
  • When you run more that one RunTests command at the same time, you must specify separate workspaces for each process to ensure reliable execution. See the -data option in the following table.
An example of using the -runPath and -runType options follows:

RunTests   [-noHTTP] [-noResultsPublishers] [-useResultsPublishers <pub_1>,<pub_2>]
[-noPublishers] [-resultsServerLogging relative] [-overrideSlowFail scenario on] 
[-projectUser <user>] [-projectPass <password>] [-loginAsAdmin] [-junitDir <directory>] 
[-securityToken <token>] [-input <file>] [-output <file>] [-environmentTags <env>] 
[-environment <name>] [-project <file>] [-runType <test>] 
[-runPath <service_component1>/<operation1>;<service_component2>/<operation2>]
An example of using the ephemeral port range in RunTests commands:

RunTests   [-environment <name>] [-project <file>] [-run <stub>] [-ephemeralPortOverrideRange "1029, 1034, 1100-1200, 1500, 1850-1920, 2000"]
The command-line parameters are described in the following table:
Table 1. Command-line parameters
Parameter Description
-data <folder> Specifies a unique folder name to store workspace files during execution. Use this option when you want to run multiple RunTests instances in parallel. For example, 

-data C:\temp\runtestsworkspace1
-environment <name> The name or ID of the Test Integrations and APIs environment to use when you run the test items.
Note: The internal ID field in the Documentation tab of the environment represents the environment ID that can be used as an alternative for the environment name.
-environmentTags

A value of env will result in all OS environment variables being mapped to environment tags, overriding tags of the same name within the project (optional).

When you use the RunTests command with this parameter, all the operating system variables and system environment variables are made available to the test run. Hence, when you run a test by using the command line without -environmentTags env, Test Integrations and APIs automatically uses the OS environment variables as tags even if you have not defined them in the test project.

Note: If you also specify -input, the properties for that argument take precedence when Test Integrations and APIs resolves an environment tag value.
-ephemeralPortOverrideRange Specifies the range of ephemeral ports to be selected when a stub is started using a transport, which opens a server socket unless explicitly overridden.
Note: A port from the range that you specify is also used if you are using the internal web server.
You can specify the different ports and different ranges of ports in the following format:
[-ephemeralPortOverrideRange "<portNumber>, <portRangeLowerBound-portRangeUpperBound>, <portNumber>, <portRangeLowerBound - portRangeUpperBound>, <portNumber>"]
For example, the ephemeral ports can be as follows: 
[-ephemeralPortOverrideRange "5800, 6500, 8000-8025, 9200, 11000-11100, 12200"]

You can also use the ephemeral port range in HCL DevOps Test Integrations and APIs Agent (Test Integrations and APIs Agent) by enabling the ephemeral port range parameter in the Agent.config file.
-input Path to an input properties file. The properties are converted to Tests Tags. For more information on Test Tags, see The Tag Data Store.
-junitDir Indicates that JUnit reports are generated for all suites that are being run and the reports are placed in the specified folder. The default is to not generate reports. For more information on the format of the output file, see JUnit style output
-junitLegacy Causes the junitDir argument to behave as it did before Test Integrations and APIs version 8.7.0. This earlier behavior requires junitDir to obtain information from the results database.
-loginAsAdmin Indicates to log in as the Administrator for a secured project. This option requires the admin password to be specified by using the -projectPass parameter.
-noHTTP An optional switch to disable the internal web server.
-noDB Causes test results to not be written to the results database, even when one is configured in the project.
-noResultsPublishers An optional switch to disable any results publishers that can be configured in the project.
-output Allows the list of tags present in the executed test, which were marked with a type of output to be written to the specified file. The contents are structured according to the Java properties file format as used by the -input argument.
-overrideSlowFail Overrides the slow fail setting in the root Scenario for a Suite. The argument has the following values:
ON
Ignore the Scenario setting and run the Suite in slow fail mode.
OFF
Ignore the Scenario setting and run the Suite in normal mode.
SCENARIO
Do not override the Scenario setting.
The default is SCENARIO.
-project <file> The full path to the Test Integrations and APIs project file <file> that contains the specified environment and test resources.
-parameterFile <file> The full path to a parameter file that contains run arguments for one or more resources. For more information, see Using the parameter file.
-projectPass The password for a secured project.
-projectUser The user name for a secured project.
-resultDatabaseDriver The driver of the results database to use to override the results database that is configured for the project in Test Integrations and APIs. For example, the driver for a MySQL database is com.mysql.cj.jdbc.Driver.
-resultDatabasePassword The password which is to be used to authenticate the user to access the results database that overrides the results database that is configured in the project in Test Integrations and APIs.
Note: You can encrypt the password by using the EncryptPassword application that is available in the installation directory of Test Integrations and APIs.
-resultDatabaseURL The URL of the results database to use to override the results database that is configured for the project in Test Integrations and APIs. For example, the URL of a MySQL database can be as follows: jdbc:mysql://localhost:3306/<database_name>.
-resultDatabaseUsername The user name that is authorized to access the results database to use to override the results database that is configured in the project in Test Integrations and APIs.
-resultLoggingLevel Specifies the volume of information that is to be written to the results database when a Test Suite is run. The argument supports the following values:
full
The content in the message headers and message body are written to the results database.
summary
A brief status of the message is written to the results database without the content in the message headers or body. This option significantly reduces the volume of data that is written to the database.
Notes:
  • You must specify a value for this argument in the RunTests command, if you want to control the volume of data that is written to the results database.
  • The value specified for this argument in the RunTests command option overrides the level of details to be written to the results database that is specified in the Logging tab for the Test Suite in the Test Integrations and APIs UI.
  • If you do not use this RunTests command option, then the level of logging that is specified in the Logging tab for the Test Suite in the Test Integrations and APIs UI is used to write information to the results database.
  • If you do not use this RunTests command option and you run a Test Suite created in Test Integrations and APIs 10.2.1 or earlier, then the content in the message headers and body are written to the results database.

When you have nested Test Suites (Test Suites that include calls to other Test Suites), the details in the results database are logged based on the logging level that you specify with the -resultLoggingLevel command option. The logging level that you specify overrides the logging level set for all the Test Suites including the nested Test Suites in the Test Integrations and APIs UI. If you do not use this command option, the Test Suites log details based on the logging level specified in the UI for each of the Test Suites.
-resultsServerLogging Specifies how the Results Server URL is written to the console when tests complete. The argument has the following values:
ignore
The report URL is not written to the console.
absolute
The full URL is written to the console.
relative
A URL relative to the current Results Server location for the project is written to the console.
The default value is absolute.
-run <res_1>;<res_2> One or more resources to be run, separated with a semicolon. A resource can be specified by any part of its name, which uniquely identifies it. You can also specify the internal resource ID instead of resource name.
Note: The internal resource ID field in the Documentation tab of the resource represents the resource ID that can be used as an alternative for the resource name.
-runPath Specify the path to the test asset you want to run. The path to the test asset must include the <service_component>/<operation> under which the asset is listed in the Test Factory view. You can specify one or more paths to the test assets that are separated with semicolons.
Note: You must use this option along with the -runType option either with or without the -run option.
Only the following combinations of commands that use the -runPath are supported:
  • -runPath <path> -runType <type>
  • -runPath <path> -runType <type> -run <res_1>
-runType Specify the type of the test asset you want to run. You can specify any of the following types: test, stub, suite, or perf. You can specify multiple assets of the same type along with their path in a single command.
Note: You must use this option along with the -run option or with the -runPath option.
Only the following combinations of commands that use the -runType are supported:
  • -runType <type> -runPath <path>
  • -runType <type> -run <res_1>
  • -runType <type> -runPath <path> -run <res_1>
-securityToken <token> Security token for authentication when domain security is enabled (optional). For more information, see Domain level security.
-secrets

Override secrets namespace defined by the environment.

when you use the RunTests command with this parameter, the secrets collection names defined by the environment is overridden and the secrets defined in the project secrets collection on HCL DevOps Test Hub (Test Hub) is retrieved.

For more information, see Accessing Test Hub from IntegrationTesterCmd, ANT CLI, Maven CLI, or in a REST API call.
-useResultsPublishers <pub_1>,<pub_2> One or more results publishers to be enabled for publishing. Each publisher is designated by the name that it was given when created in Test Integrations and APIs. Multiple publishers must be separated by commas.
Note: To learn about using the EncryptPassword application, see Configuring the security settings after installation by updating the security.config file.

Using the parameter file

You can set the project, environment, and run arguments in a parameter file, which you specify by using the parameterFile argument.

You can specify the run arguments for one or more test resources, which are grouped by project and separated by a semicolon. As an example, consider a file that is named myparams.txt that has the following contents:


-project C:\Projects\AETesting\myproj.ghp -environment Env1 -run "creditTest1";"creditTest2";
-project C:\Projects\BWTesting\myproj.ghp -environment HotelApp -run "bookVisa;bookMaestro;reg Suite1;reg Suite2";

You can use the parameter file in the RunTests command, as follows:

RunTests -noHTTP -resultsServerLogging ignore -parameterFile C:\Projects\myparams.txt

In the example file, the command runs the following resources:

  • In the AETesting project, creditTest1 resources are run followed by creditTest2, and both are run in the Env1 environment.
  • Next, in the BWTesting project and by using the HotelApp environment, bookVisa, bookMaestro, reg Suite1, and reg Suite2 resources are run (in that order).
Note: A final trailing semicolon is used in the parameter file to close the list of artifacts to run.

Checking the test status

After you run the script, you can check the test status by running the following commands:
  • On Windows™ systems, echo %errorlevel%
  • On systems other than Windows™, echo $?
The return code 0 indicates that the test passed and 1 indicates that the test failed.