Creating a pipeline script from Jenkins

You must create a pipeline script to run test assets from the Jenkins Pipeline project.

Before you begin

You must have completed the following tasks:

Procedure

  1. Open your Jenkins Pipeline project from the list.
  2. Click Configure, and then select the Pipeline tab.
  3. Click Pipeline Syntax to generate a pipeline script.
  4. Select the step: General Build Step option from the drop-down list in the Sample Step field.
  5. Select the Run HCL DevOps Test UI - UI Test option from the drop-down list in the Build Step field.
  6. Provide the details about the test run for the fields as listed in the following table.
    You must provide the values for the following required fields:
    • Name
    • Workspace
    • Project
    • Test Suite Name
    If you include the required parameters such as Workspace, Project, and Test Suite Name in a configuration file and use the Config File field to specify the complete file path, then the values for these three fields are not required.
    Field Description
    Required
    Name

    The name of the Jenkins build step.
    Workspace

    The complete path to the Eclipse workspace.
    Project

    The path, including the file name of the project relative to the workspace.
    Test Suite Name

    The path, including the file name of the test to run related to the project.

    Note: You must provide the file name along with the file extension if you are using an Accelerated Functional Test suite.

    To run multiple tests of the same project sequentially, you must specify the test names separated by a comma.
    Config File

    The complete path to a file that contains the parameters for a test run.

    For example, E:\Workspace1\Project1\Tests\config_file1.txt

    For more information about creating command line config file, see Creating a command-line config file.

    Remember: If you include the required parameters such as Workspace, Project, and Test Suite Name in a configuration file and use the Config File field to specify the complete file path, then the values for these three fields are not required.
    Optional
    IMShared Location

    The complete path to HCLIMShared location, if it is not the default location.
    Import The complete path to the project folder that is cloned from a source control system or remote repository. When you use this option, you can specifiy a name for the new workspace to be created to run the imported test assets.
    Var File

    The complete path to the XML file that contains the variable name and value pairs.
    Results File

    The name of the results file.

    The default result file is the test or schedule name with a time-stamp appended. The results file is stored in the Results directory. If you are running multiple tests, do not provide a name for the results file.
    Overwrite Results File

    Determines whether a result file with the same name is overwritten. The default value is true, which means the result file can be overwritten.
    VM Args

    The option to pass Java™ virtual machine arguments.
    Protocol Input
    The option to customize the test run:
    • Run a Web UI test in parallel on different browsers.

      For example, Protocol Input is all.available.targets.in.parallel=all or

      Protocol Input is all.available.targets.in.parallel=chrome,ff,ie

      Note: If you use the Protocol Input argument, you must not use the equivalent VM Args arguments:
      VM Args = -Dall.available.targets.in.parallel=all
VM Args = -Dall.available.targets.in.parallel=browser1,browser2,browser3
    • Run a UI test in the background through an agent.

      You can use bg=true to run a UI test in the background. The test execution continues even when the screen is locked or the agent is disconnected.
    Dataset Override

    For a test, the default value is the dataset specified in the test editor.

    Note: You must use the Dataset Override option to replace the dataset values during a test run. You must ensure that both original and new datasets are in the same workspace and have the same column names. You must also include the path to the dataset.

    For example,

    /project_name/ds_path/ds_filename.csv:/project_name/ds_path/new_ds_filename.csv

    You can swap multiple datasets that are saved in a different project by adding multiple paths to the dataset separated by a semicolon (;).
    Labels

    The option to adds labels to test results when the test run is complete.

    You can add multiple labels to a test result separated by a comma.

    For example, label1, label2

    When you run test assets then the same labels are displayed on the UI Test Statistical Report in Test UI.

    If you have set Publish result after execution as Always or Prompt in the Test UI preferences (Window > Preferences > Test > DevOps Test Hub) and use the Labels option, then the Results page of DevOps Test Hub displays the same label for the specific test asset.

    Note:
    • When you run tests by using the double quotation marks ("") for the Labels field, then the labels in the test result do not include the double quotation marks.
    • To work around this problem, you must create a command-line config file, and then run the test by using the Config File field.
    • When you use the Config File field to run tests, then labels provided in the configuration file take precedence over the labels provided in the Labels field.
    Exported Statistical Report Data File

    The complete path to a directory to store exported statistical report data.

    If you do not supply a value for Exported Statistical Report Data File, the logs will be saved in Jenkins workspace/temp directory.
    Custom Report Format Files

    A comma-separated list of absolute paths to custom report format files (.view files) to use when exporting statistical report data with the Export Statistical Report Data File option.
    Exported Statistical Report in html

    The complete path to a directory to export web analytic results. Analyze the results on a web browser without using the test workbench. If you run multiple tests, do not provide a value in this field. The web analytic results will be exported to the Jenkins workspace.
    User Comments

    The text to be displayed in the User Comments row of the report.

    Add text within the double quotation mark ()

    Note:
    • When you run tests by using the double quotation marks ("") for the User Comments field, then the User Comments row of a report does not contain double quotation marks.
    • To work around this problem, you must create a command-line config file, and then run the test by using the Config File field.
    Export stats format

    The field to specify a format for the report that you want to export. You must use the Exported Statistical Report Data File parameter along with the Export stats format parameter.

    You must use at least one of the following formats:

    • simple.csv

    • full.csv

    • simple.json

    • full.json

    • csv

    • json

    For example, 
    Exported Statistical Report Data File = <local_dir_path> 
Export stats format = json

    You can add multiple formats for the report separated by a comma (,).

    If you want to export both the simple and full reports in a json or csv format, you can specify json or csv as the format in the field.

    The reports are saved to the location specified in the Exported Statistical Report Data File field.
    Export Report The option to export the unified report of UI tests to the file formats such as PDF, HTML, and XML.
    Note: The exported XML file is a JUnit XML file. You can view this file in applications that support JUnit reporting formats.
    After you select this checkbox, you must choose the following details from the drop-down lists and enter the details:

    • Type: Select unified from the list.

    • Format: Specify the supported file formats such as xml, pdf, and html to export the unified report. You can also specify multiple formats separated by a comma to export the unified report simultaneously to different file formats.

      For example, to export the report to all the supported formats, you can use the following values:

      pdf,xml,html

    • Directory: Enter the directory path where you want to save the exported file.

    • File Name: Enter a name for the exported file.

    Remember: You must provide the values in the required fields such as Workspace, Project, and Test Suite Name to export the unified report. You must ensure not to use the Config File parameter with the Export Report parameter.
    Publish The option to publish test results to DevOps Test Hub.

    You must provide the URL and offline user token of the server in Window > Preferences > Test > DevOps Test Hub of Test UI before you use the Publish parameter in the test script.

    Use the following arguments with the Publish parameter:

    • To specify the project name, use any of the following formats:

      • serverURL #project.name=projectName&teamspace.name=name_of_the _teamspace

      • serverURL #project.name=projectName&teamspace.alias=name_of_the _teamspace_alias

      You must consider the following points while providing the project name:

      • If the project name is not specified, then the value of the Project parameter is used.

      • If you have a project with the same name in different team spaces, then you can append either the &teamspace.name=name_of_the _teamspace or &teamspace.alias==name_of_the_teamspace_alias options.

        For example: 
        Workspace = C:/Users/IBM/devopstest/workspace1
Project = proj1
Test Suite Name = Tests/testHttp.testsuite
Publish = https://localhost:5443#project.name=test&teamspace.name=ts1
        Where:

        • https://localhost:5443 is the URL of the server.

        • test is the name of the project.

        • ts1 is the name of the team space.

      • If the name of the project or team space contains a special character, then you must replace it with %<Hex_value_of_special_character>.

        For example, if the name of the team space is Initial Team Space, then you must provide it as Intial%20Team%20Space.

        Where, %20 is the hexadecimal value of the space character.

    • To avoid publishing of reports, use no.

      You can use the no option if you do not want to publish test results after the run. This option is useful if the product preferences are set to publish the results, but you do not want to publish them.

      For example:
      Workspace = C:/Users/IBM/devopstest/workspace1
Project = proj1
Test Suite Name = Tests/testHttp.testsuite
Publish = no

    If you do not use the Config File parameter to run the tests, then the values provided in the test script always take precedence over the Results options set in the product preferences (Window > Preferences > Test > DevOps Test Hub > Results).

    The Reports information section on the Output window displays the names of the report along with its corresponding URLs in the following conditions:

    • When you configured the URL of DevOps Test Hub in preferences of Test UI (Window > Preferences > Test > DevOps Test Hub.

    • When you set Publish result after execution as Always or Prompt in the preferences of Test UI (Window > Preferences > Test > DevOps Test Hub > Results).
    PublishFor

    The option to publish test results to DevOps Test Hub based on the completion status of the tests. You must use the PublishFor parameter along with the Publish parameter.

    The following are the available options that you can use for the PublishFor parameter:

    • ALL - This is the default option. You can use this option to publish test results for any text execution verdict.
    • PASS - You can use this option to publish test results for the tests that have passed.
    • FAIL - You can use this option to publish test results for the tests that have failed.
    • ERROR - You can use this option to publish test results for the tests that included errors.
    • INCONCLUSIVE - You can use this option to publish test results for the inconclusive tests.
    You can add multiple parameters separated by a comma.
    For example: 
    Workspace = C:/Users/IBM/devopstest/workspace1
Project = proj1
Test Suite Name = Tests/testHttp.testsuite
Publish = https://localhost:5443#project.name=test&teamspace.name=ts1
PublishFor = FAIL,ERROR
    Publish reports The option to publish test results in DevOps Test Hub. The values that you can use with Publish reports are as follows:
    • FT - This is an identifier for Functional Test Report. You can use this value to publish the unified report if it is available for the selected test. See Unified reports.
    • STATS - This is an identifier for Statistics Report. You can use this value to publish the web analytics report if it is available for the selected test. See UI Test Statistical report.
    • TESTLOG - This is an identifier for Test Log. You can use this value to publish the test log if it is available for the selected test. See Logs overview.

    You must use the Publish reports parameter along with the Publish parameter.

    For example: 
    Workspace = C:/Users/IBM/devopstest/workspace1
Project = proj1
Test Suite Name = Tests/testHttp.testsuite
Publish = https://localhost:5443#project.name=test&teamspace.name=ts1
Publish reports = STATS, TESTLOG

    The values specified here override the values selected in Window > Preferences > Test > DevOps Test Hub > Results of Test UI.

    You can prefix the value with ! to publish the reports except for the specified one in the test script.

    For example, 
    Workspace = C:/Users/IBM/devopstest/workspace1
Project = proj1
Test Suite Name = Tests/testHttp.testsuite
Publish = https://localhost:5443#project.name=test&teamspace.name=ts1
Publish reports = !TESTLOG

    All the reports except the TESTLOG report is published to DevOps Test Hub after executing the command.
  7. Click Generate Pipeline Script and copy the script that you created for the build step to use it in the sample script.
    Note:

    You need this generated script to enter in step 5.c in the following topic:

    Configuring the Pipeline project by using the sample script

Results

You have created the pipeline script.

What to do next

You can configure the Pipeline project by using the Pipeline script option. See Configuring the Pipeline project by using the sample script.