Running multiple suites or tests by using the wildcard pattern

When you want to enable a single suite to run multiple tests and suites based on their name, you can use a Run step to refer to tests and suites by using wildcard patterns. You can also use these patterns to specify the suites that must be excluded.

Before you begin

You must have completed the following tasks:
  • Ensured that you are assigned a Team Space Owner, Project Owner, or Member with a Tester role to create or edit a test resource.
  • Read and understood the information in Creating a suite.
  • Ensured that there are multiple suites available in your project.

About this task

Within a Run step, wildcard patterns can be used to reference tests and suites. The following two wildcards are supported:
  • Single Asterisk (*): Matches zero or more characters within a single folder and it does not match the / character.
  • Double Asterisk (**): Matches zero or more characters including the / character. It can be used to match across multiple folder.

You can only match files located in the same folder as the test suite, or within its subfolders. You cannot use patterns to match files located in a different folder for a different application.

The following table lists the sample patterns:
Pattern What it matches
*.dtx.yaml All tests located in the same folder as the test suite.
a*.dtx.yaml All tests starting with the letter "a" in the same folder as the suite.
**/*.dtx.yaml All tests in subfolders under the folder containing the suite.
**/abc/* All tests and suites located in any subfolder that is named ‘abc’ under the folder containing the suite.

Procedure

  1. Log in to Test Hub.
    The Projects page of the initial team space is displayed.
  2. Click My projects > project_name to open the project that contains the test assets.
    The Overview page of the project is displayed.
  3. Click Author > Test Editor.
    The Test Editor page and the test navigator panel are displayed.
  4. Perform one of the following actions:
    • Create a new suite, open the suite, and then add a test or suite.
    • Open a suite from the list of Suites.
    The contents are displayed on the Suite Editor page.
  5. Click a Run step.
    The step details pane is displayed.
  6. Enter a wildcard pattern in Path to specify the tests or suites to be included in the test run.
  7. Click +Add Exclusion if you want to exclude some of the suites.
    A new text box is displayed.
  8. Perform one of the following actions to specify the suite or test to be excluded from the test run:
    • Enter the file name.
    • Click the Browse for suite or test button to browse and select the suite, and then click Add.
    • Drag a suite or test from the test navigator panel to Suite Editor.
    • Enter a wildcard pattern.
  9. Click Close to close the step details pane.
  10. Optional: Click the Configure settings and data step, and then select the Run in parallel checkbox to enable multiple tests to run at the same time.

    Tests that are resolved by a wildcard run in parallel only if the suite contains a single run step. That is, the only step defined in the suite must be the test with the wildcard expression.

  11. Click Save to save the changes.
  12. Click the Run all icon icon, and then select the Run all > Run all on Default Cluster option.

    You can also select the Run all on Personal Agent option if the personal agent is in a ready state.

    The Results tab displays the progress of the test run. After the test run is complete, the tab displays the verdict, and links to view the functional report, test log, and results page.

Results

You have run multiple suites or tests.

What to do next

You can view the following information:

  • The test results, reports, and logs from the Results page after the test completes the run.
  • The summary of the test run, which includes passed or failed status, count of passed or failed tests or test steps, and the verification points in the functional report. See Viewing a Functional Report.