Test Suites: Overview and tasks

A Test Suite represents a group of tests, test suites, and scenarios. Any combination of these items can be included, but a test suite must always contain at least one scenario and one test. Test suites are used mainly to construct reusable, packaged regression tests that can be run across multiple environments, or repeatedly within the same environment.

A test suite can be added to an existing operation or to an existing test suite folder, either from the menu or the Test Factory toolbar. After you provide a name for it, the test suite is opened in the Test Suite Editor in the Test Factory view.

You can find the following tabs for a Test Suite in the Test Suite Editor:
Image of the test suite editor

The Config tab

You can add a scenario or test to the Test Suite from the Config tab.

To add a scenario, click the Add scenario icon.

To add a test, click the Add tests icon.

When a test suite is first created, it is empty except for a scenario. The scenario is a container that controls how the items contained in the scenario are to be run.

For more information about scenarios, see Test suite scenarios.

A Test Suite can contain a single or multiple number of tests, Test Suites, or scenarios, in any combination.
Note: When a Test Suite is run, the tasks are processed from the top to the bottom as displayed in the Test Suite Editor.

The Continue on Fail option is selected as the default option for all items added to a Test Suite. For more information, see The Continue on Fail option.

The Logging tab

You can specify how much of information about the tests or stubs that are to be written to the results database in the Logging tab of a Test Suite.

Image of the logging tab of a test suite
The following table describes the list options that are available to set the configuration for writing of information to the results database when a Test Suite is run:
Option Description
Full The content that is in the message headers and message body are written to the results database. This is the default option.
Note: If the Full option is selected, it might result in a large volume of the storage capacity of the results database to be used when the number of tests, Test Suites, or scenarios are many in a Test Suite or when the Test Suite is run multiple times.
Summary The content that is in the message headers is written to the results database. This option reduces the volume of data that is stored in the results database.
The following table describes the list options that are available to set the configuration for writing of information to the results database when a stub that is contained in a Test Suite is run:
Option Description
None No information is logged.
Normal Standard information is logged. This is the default option to write the information about the stubs to the results database.
Debug Verbose information is logged.
Note: If the Debug option is selected, it might result in a large volume of the storage capacity of the results database to be used when the number of stubs are many in a Test Suite or when the Test Suite is run multiple times.

Optionally, you can opt to free up the space on the results database be removing the information about messages sent or received after the Test Suite completes its run by selecting the checkbox in the Messages panel. This setting overrides the option specified for the children tests. If the Test Suite is connected to the database, the messages for tests that passed can still be viewed in the Results Gallery.

When you have nested Test Suites (Test Suites that include calls to other Test Suites), the details in the results database are logged based on the logging level that you configured for each of the Test Suites.
Note: If you use the command line to run nested Test Suites and specify the logging level with the -resultLoggingLevel command option, the logging level that you specify overrides the logging level set for the Test Suites including the nested Test Suites.

The Documentation tab

You can opt to enter additional information about the Test Suite in the Description or the External ID fields, which help you to identify the Test Suite when there are multiple Test Suites or instances of the same Test Suite run.Image of the Documentation tab

The Publish Results tab

You can opt to associate a Results Publisher that is configured in your project in Test Integrations and APIs, so that the results of the Test Suite are automatically published. You must set up and configure a Results Publisher before you can select it in the Publish Results tab of the Test Suite. See Results publishers.