Running a suite concurrently

You can run a suite to run multiple tests and other suites in parallel to expedite the testing process.

Before you begin

You must have completed the following tasks:
  • Ensured that you are assigned a Team Space Owner, Project Owner, or Member with a Tester role to create or edit a test resource.
  • Ensured that a suite is available in your project.
  • Added tests or other suites to the suite.

About this task

You can run a suite concurrently to run multiple suites and tests simultaneously across several pods. You can set up the maximum number of concurrent pods, which defines how many suites or test assets can run in parallel. In the default set up, suites run their child assets sequentially on a single pod, with only one test asset running at a time.

When you select the Run in Parallel option, the Continue on Fail checkbox is disabled. Whether a test passes or fails, all test assets run independently and concurrently across the available pods.

Procedure

  1. Log in to Test Hub.
    The Projects page of the initial team space is displayed.
  2. Click My projects > project_name to open the project that contains the test assets.
    The Overview page of the project is displayed.
  3. Click Author > Test Editor.
    The Test Editor page and the test navigator panel are displayed.
  4. Click the .dts.yaml file in the list of Suites to open the suite.
    The tests or suites are displayed on the Suite Editor page.
  5. Click the Configure settings and variables step.
    The Settings pane is displayed.
  6. Select the Run in parallel checkbox to enable the test to run in parallel and multiple tests to run at the same time.
    The Continue on Fail option gets disabled.
  7. Enter the value in the Maximum Concurrent Pods field.
    Note:

    The Maximum Concurrent Pods setting defines the number of test assets from a suite that can run in parallel. A lower value reduces server resource usage but increases run time, and a higher value speeds up the run time at the cost of higher resource consumption.

  8. Click the Try Now icon try now icon, and then select the Run all option.
    You can view the following stages on the progress bar during the test run:
    • In transition: The test setup is in progress.
    • Running: The test is actively running.
    • Complete: The test run is completed.
    • During the Running phase, you can view the following details:
      • A checkmark check mark icon after each test or suite that passes, and a crossmark cross mark icon for any failed test or suite.
      • The Results tab displays the test as In Progress during the test run.
    • After the test run is completed, you can view the following results:
      • The Results tab displays the test as Pass or Fail.
  9. Click Analyze to open the Results page, where you can view the results, reports, and logs of the completed test run, and perform the following steps:
    1. Verify that a secondary label appears for the secondary executions.
    2. Click on secondary and main/primary execution.
    3. Verify that the Test Log and Functional Report options are displayed only for the main/primary execution and does not display for the secondary executions.

Results

You have run a suite concurrently.

What to do next

You can view the following information:

  • The details of the suites and individual test results from the Run Details tab.
  • The console output, which includes error messages and logs. See Accessing console output.
  • The test results, reports, and logs from the Results page after the test completes the run.
  • The summary of the test run, which includes passed or failed status, count of passed or failed tests or test steps, and the verification points in the functional report. See Viewing a Functional Report.