Running DevOps Test Hub tests in an Azure DevOps Pipeline

After you install the HCL DevOps Test Hub extension in your organization, you can run tests that are available in HCL DevOps Test Hub (Test Hub) as a job in Azure DevOps pipelines.

Before you begin

You must have completed the following tasks:

About this task

After you add the HCL DevOps Test Hub extension in your Azure DevOps organization, you can use an existing pipeline or create a new one to add Test Hub test tasks. You can install an agent or use the one that you installed in your default agent pool. You can add Test Hub tests as tasks to your agent job, configure the task, and then run the task in the Azure DevOps pipeline.

If you have created test cases under test plans in a project on Azure DevOps, you must provide the details of the Azure DevOps URL, test plan, test case, and your PAT while you configure the test job in a pipeline so that you can view the results of the test run on your Test Plan dashboard.

Procedure

  1. Open your Organization page in Azure DevOps and perform the following steps:
    1. Click the project you want to use.
    2. Initialize the repository by performing the following steps:
      1. Click Repos from the left pane.
      2. Click Initialize from the Initialize with a README or gitignore section.
        Note: Select the Add a README checkbox if it is not selected.
    3. Click Pipelines from the left pane.
    4. Click Create Pipeline.
    5. Click Use the classic editor to create a pipeline without YAML.
    6. Verify the project, repository, and branch for manual and scheduled builds, and then click Continue.
    7. Click Empty job.
  2. Select Pipeline and complete the following steps:
    1. Change the name for the build pipeline if required.
    2. Select the Agent pool for your build pipeline.

      You can use the agent from the default agent pool or use the one you have installed.

    3. Select the Agent Specification for the agent if required.
  3. Add a task to the agent job by completing the following steps:
    1. Click the Add Task icon Add Task icon in Azure DevOps for the agent job.

      The Add tasks pane is displayed.

    2. Search for the HCL tasks defined in the HCL DevOps Test Hub extension.

      The tasks that you can select are displayed.

    3. Select the HCL DevOps Test Hub Task, and then click Add to add the task to the agent job.

      The selected task is added to the agent job and it is displayed with a warning that some settings require attention. You must configure the settings mentioned in Step 4.

      You can also remove the tasks that are not required in your job. Select the tasks in the list that you want to remove. You can then right-click the tasks, and click Remove selected task(s) to remove them.

  4. Configure the settings by performing the following steps:
    1. Select the task version from the list if required.
    2. Follow the action for the task by referring to the following table:
      Note: All mandatory fields are marked with an asterisk (*) in the UI.

      The following table provides the options for the task that you must configure.

      Field

      Description

      Action

      Display name

      The default name of the task. The default name that is displayed is HCL DevOps Test Hub Task.

      Enter a name for the task that helps you to identify the task by changing the default name.

      HCL DevOps Test Hub Task service connection

      The details of Test Hub to set up a connection with Azure DevOps.

      Select the service connection from the drop-down list.

      If you are selecting the Test Hub service connection for the first time, you must click New to add the following details to add Test Hub service connection, and then save the connection details:
      • Server URL - Enter the URL of Test Hub. The format of the URL is as follows: https://hostname.
      • Offline Token - Enter the offline token that you generated from Test Hub.
      • Service connection name - Enter a service connection name.
      • Description (optional) - Enter the details of the service connection if required.
      • Optionally, select the Grant access permission to all pipelines checkbox if required.
      You can save the service connection details. The service connection is available for selection from the Test Hub service connection drop-down list.
      Note:

      You can edit or delete the service connection that you added if required. Click Manage, and then select the service connection from the Service connections list to open the service connection. You can click Edit to edit the service connection details. If you want to delete an existing service connection details, click the Vertical ellipsis icon, and then click Delete to delete the service connection details.

      Optionally, you can also create, edit, and delete a service connection from your Azure DevOps project dashboard. Open your project, click Project settings > Service connections, and then perform any of the following tasks:
      • Create a new service connection.
      • Edit or delete an existing service connection that you already added.

      Team Space Name

      		 The name of the team space on Test Hub that contains your project with the test assets.
      Enter the name of the team space that contains the project.
      Note: The license on the team space must be configured and you must be a member of that team space.

      Project

      The name of the project containing the test assets in your team space on Test Hub.

      Enter the name of the project from the available names in the team space.

      Note:
      Note: The Project drop-down list displays the projects where you are an Owner or a member of Test Hub. You must be an Owner or a Tester of the project that is available in the team spaces to run the tests from the Jenkins server.
      Azure DevOps pipeline.

      Branch

      		 The name of the branch of the repository in your project on Test Hub that contains the test assets.

      Enter the branch where the test assets are stored.

      Repository Link

      		 The URL or link to the repository in your project that contains the test assets.

      Enter the repository path that is configured for the project that contains the test to run.

      File Path

      		 The path of the test in the repository of your project. Enter the path of the test in Test Hub that you want to run.

      API Test Environment

      The environment defined in the API Suite or test to use in the test run.
      Enter the name of the environment that is configured in the test asset for the test created in Test Integrations and APIs.
      Note: This field is applicable only if you are running an API suite test.

      Variables

      		 The variable name and its value. Enter the name of the variable and its value if your test requires variables during the test run time.

      You must enter the variables in the following format:

      name_of_the_variable=value_of_the_varibale

      You can add multiple variables to the test run separated by a semicolon.

      For example, varname1=value1;varname2=value2

      Datasets

      		 The name with path to the dataset that must be used during the test run. Enter the path to the dataset if you want to replace the values of the dataset during a test run.

      You must ensure that both original and new datasets are in the same workspace and have the same column names. When you enter a value for the Datasets field, you must also include the path to the dataset. The path must be in the following format:

      /project_name/ds_path/original_ds.csv:/project_name/ds_path/new_ds.csv

      Note: You can override multiple datasets that are saved in a different project by adding multiple paths to the dataset separated by a semicolon.

      Labels

      		 The option to add labels to the test results when the test run is complete. Enter the labels to add to test results when the test run is complete.

      You can add multiple labels to a test result separated by a comma.

      For example, label1, label2

      After the test run is complete, then the values provided in the Labels field are displayed on the Results page of Test Hub.

      Classification Name

      		 The name of the classification that contains the secrets to be used in certain API Suites or tests. Enter the name of the classification that contains the secrets to be used in the API Suite or test.
      Note: This field is applicable only if you are running an API suite test.

      Azure DevOps Project URL

      The URL of the test project in the organization on the Azure DevOps server.

      Note: You must enter the details for this option if you have created test cases under test plans in your Azure project and want to view the test results on the Test Plans Dashboard.
      		 Enter the URL of the test project in the organization on the Azure DevOps server in the following format:

      https://<host>/<orgname>/<projectName>

      You must use this option with the following options:
      • Azure DevOps PAT
      • Test Plan Name
      • Test Case Name

      Azure DevOps PAT
      The personal access token (PAT) of the Azure server where the test project that contains the test plan is hosted. You can also enter the variable for the encrypted PAT that you create in Azure DevOps.
      Note: The token must have read and write access.
      		Enter your personal access token.
      You must use this option with the following options:
      • Azure DevOps Project URL
      • Test Plan Name
      • Test Case Name

      Test Plan Name

      The name of the test plan in your Azure project.

      		 Enter the name of the test plan.
      You must use this option with the following options:
      • Azure DevOps Project URL
      • Azure DevOps PAT
      • Test Case Name

      Test Case Name

      The name of the test case under the test plan in your Azure project.

      		 Enter the name of the test case.
      You must use this option with the following options:
      • Azure DevOps Project URL
      • Azure DevOps PAT
      • Test Plan Name

      Export JUnit Report

      		 The location to store the JUnit report of the test. Enter the path to store the JUnit report that is generated for the test in the XML format.

      You must specify the complete path to the directory including the file name of the report.

      For example, C:/Users/Documents/Reports/junit_test_report.xml
    3. Expand Control Options and configure the settings for your task if required.
    4. Expand Output Variables and configure the settings for your task if required.
  5. Select from the following options:
    1. Click Save to save the configured settings for the task.
      Note: The task is not queued for a run.

      You can save the task to a build pipeline and opt to run the build at a later time.

    2. Click Save & queue to save the configurations and queue the run in the pipeline.

      The Run pipeline dialog is displayed.

  6. Complete the following steps:
    1. Enter a comment for the test in the Save comment field.
    2. Select the agent that you configured for the test from the Agent pool list.
    3. Select the agent specification from the Agent Specification list if required.
    4. Select the branch from the Branch/tag list.
    5. Add the variables and demands for the task run from the Advanced Options pane if required.
    6. Select the Enable system diagnostics checkbox for a detailed log view.
    7. Click Save and run.

      The pipeline summary page displays the progress of the job run.

Results

You have run the tests for the application that you are testing in the Azure DevOps pipeline.

What to do next

You can perform any of the following actions in Azure DevOps:
  • View the test results on the Task page by clicking the task.

    The Reports information section on the Task page displays the names of the report along with its corresponding URLs. The report URLs are the Test Hub URLs where the reports are stored. You can access the report URLs to view the test run information at any point of time.

  • View the task logs from the pipeline summary page for the job you ran.
  • View the test results on the Test Plans Dashboard if you created test cases under test plans in your Azure project.
You can perform any of the following actions in Test Hub:
  • View the test results of the test that you ran from Azure DevOps on the Results page.
  • View the logs and reports of the test that you ran from Azure DevOps by clicking the result panel of the test asset or resource, and then click the log or the report you want to view.