Configuration of test runs from the command line

You must not use the -suite option along with the other options.

Starting from 9.2.1.1, you can execute multiple tests simultaneously.

For example, -suite test1:test2:test3.

You can run any or all tests that are in a folder by using the wildcard character * enclosed within quotes in different combinations in the command line. The following command runs the Web UI tests, compound tests, and Accelerated Functional Tests in the project folder:

cmdline -workspace <workspace_full_path> -project <project_relative_path> -suite "*"

For example, aftsuite="aftinput"

You can run the following command to run all the AFT tests that have file names starting with 'AFT': cmdline -workspace <workspace_full_path> -project <project_relative_path> -aftsuite "AFT*"

For example, cmdline -workspace "workspace" -project "UI1" -aftsuite "AFT*"

For example, aftsuite="aftinput"

You can use this option to provide the complete path of eclipse.exe.

For example, to export an HTTP performance report, specify http.

To copy the report IDs list into your command line, navigate to Window > Preferences > Test > Performance Test Reports > Export Reports. Under Select reports to export, select the required reports, and click Copy ID to clipboard. You can then paste the clipboard content on to your command line editor.

Starting from 10.0.1, by using the -exportlog parameter, you can provide multiple parameter entries when running multiple tests. You must use a colon to separate the parameter entries.

For example: -exportlog c:/logexport.txt:c:/secondlogexport.txt

If there are multiple -suite option entries with a single -exportlog parameter entry, then the -exportlog option generates the appropriate number of test logs by appending 0, 1, 2, and so on to the -exportlog option entry name.

For example: -suite "sampletest1:sampletest2:sampletest3" -exportlog c:/logexport.txt

The command generates the following test logs:

The last test log generated has the same name as that of the initial -exportlog entry.

The command syntax is as follows:

exportReport "type=<reporttype>;format=<file type1,file type2,file type3>;folder<destination folder path>;filename=<name of the exported file>"

For example, to export the report to only the pdf format, you can use exportReport "type=unified;format=pdf;folder=Exportedreport102;filename=testreport

If you want to export the report to multiple formats, you can specify the file formats as comma-separated values. The file type value can be in uppercase or lowercase.

For example, to export the report to all the supported formats, you can use

exportReport "type=unified;format=pdf,xml,html;folder=Exportedreport102;filename=testreport

You can exclude the filename attribute in the exportReport option so that the reports are exported with the respective test suite name. For example, when you run multiple tests by using a single command, if you do not specify any file name in exportReport, then the name of the test is used for the exported report name, which makes it easy to identify reports.

To copy the report IDs list into your command line, navigate to Window > Preferences > Test > Performance Test Reports > Export Reports. Under Select reports to export, select the required reports, and click Copy ID to clipboard. You can then paste the clipboard content on to your command line editor.

• simple.csv

• full.csv

• simple.json

• full.json

• csv

• json

For example, -exportstats <local_dir_path> -exportstatsformat simple.json

You can add multiple arguments separated by a comma.

For example, -exportstats <local_dir_path> -exportstatsformat simple.json, full.csv

When you want to export both simple and full type of test results in a json or csv format, you can specify json or csv as the arguments in the command. When the test run completes, the test result exports to simple.json and full.json files.

For example, -exportstats <local_dir_path> -exportstatsformat json

You can select the Command Line check box from the product preferences (Window > Preferences > Test > Performance Test Reports > Export Reports) when you want to export test results to one of the selected formats after the test run completes.

For example, when you select the Command Line option and Report format to json in the product preferences, and run the test from the command-line interface without using the -exportstats option. The result is exported to a json file after the test run is complete.

You can use this option along with the -exportstats option to specify a pattern that defines the name of a csv or json file. You can use the following token to customize the file name: "{TEST_NAME}{AGENT_NAME}{SPLIT_INDEX}_{TIMESTAMP}_{REPORT_ID}_{EXPORT_TYPE}"

For example,

cmdline -workspace "C:\Users\workspace" -project "proj123" -suite "test1.testsuite -exportstatshtml" C:\Users -exportstatsformat "simple.json,full.csv" -exportstats C:\Users -exportlog "C:\Users\logexport.txt" -exportstatreportlist "http,httpVP" -exportstatspattern "RFT_{REPORT_NAME}_{EXPORT_TYPE}_{REPORT_ID}"

• jaeger: To send test logs to the Jaeger UI during the test run.
• testlog: To send test logs as traditional test logs in Test UI during the test run.
• null: To send no test logs either to the Jaeger UI or Test UI during the test run.

-workspace workspace_full_path -project proj_rel_path 
-suite suite_rel_path -stdout -history comma delimited list of modes

-workspace C:/Users/HCL/devopstest/test_ws 
-project Project1 
-suite test1.testsuite -stdout -history jaeger

-workspace C:/Users/HCL/devopstest/test_ws 
-project Project1 
-suite test1.testsuite -stdout -history jaeger, testlog

You can add multiple labels to a test result separated by a comma.

For example, -labels label1, label2

You can also use the -labels option along with the -publish option to add labels to a test result when you want to publish test results to DevOps Test Hub.

When you run test assets from the command-line interface by using the -labels option, then the same labels are displayed on the UI Test Statistical Report in Test UI.

Similarly, when you use the -labels option with the -publish option from the command-line interface, then the Results page of DevOps Test Hub displays the same label for the specific test asset.

Determines whether a result file with the same name is overwritten. The default value, false, indicates that the new result file is created. If the value is true, the file is overwritten and retains the same file name. You must use double quotes for values true or false.

Required. This option is required only if the folder is at a different location.

• To run a Web UI test in parallel on different browsers

  -protocolinput "all.available.targets.in.parallel=all"

  -protocolinput "all.available.targets.in.parallel=chrome,ff,ie"

  Note: If you use the -protocolinput argument, you must not use the following equivalent -vmargs arguments:

  -vmargs "-Dall.available.targets.in.parallel=all"
  -vmargs "-Dall.available.targets.in.parallel=browser1,browswer2,browser3"

• To specify the Web UI preferences such as highlighting the page element and capturing screens

  For example, -protocolinput "webui.highlight=<value>;webui.report.screenshots=<value>" where webui.highlight specifies whether the page element must be highlighted and webui.report specifies whether the screens must be captured while playing back the test in the browser.

• To run only the failed tests from a previous playback in an Accelerated Functional Test suite

  cmdline -workspace workspacename -project projectname -aftsuite aftsuitname -exportlog exportlogpath -results autoresults -protocolinput "runfailedtests=true"

  In the preceding example, runfailedtests=true specifies whether the failed test from a previous playback must be rerun in Accelerated Functional Test suite.

• To automatically resolve the browser and driver incompatibility, while you play back the Web UI tests

  -protocolinput "webui.browser.driver.autoupdate=true"

• To apply guided-healing and self-healing features while you run Web UI tests

  cmdline -workspace workspacename -project projectname -suite test1 -exportlog exportlogpath -results autoresults -protocolinput "autoheal=true"

You can use the following options along with the -publish parameter:

• no

  You can use the no option if you do not want to publish test results after the run. This option is useful if the product preferences are set to publish the results, but you do not want to publish them.

• You can use any of the following options to specify the project name:

  • serverURL #project.name=projectName&teamspace.name=name_of_the _teamspace

  • serverURL #project.name=projectName&teamspace.alias=name_of_the _teamspace_alias

  You must consider the following points while providing the project name:

  • If the project name is not specified, then the value of the -project parameter is used.

  • If you have a project with the same name in different team spaces, then you can append either the &teamspace.name=name_of_the _teamspace or &teamspace.alias==name_of_the_teamspace_alias options along with the -publish parameter.

    For example: -publish “https://localhost:5443/#project.name=test&teamspace.name=ts1”

    Where:

    • https://localhost:5443 is the URL of the server.

    • test is the name of the project.

    • ts1 is the name of the team space.

• ALL - This is the default option. You can use this option to publish test results for any text execution verdict.
• PASS - You can use this option to publish test results for the tests that have passed.
• FAIL - You can use this option to publish test results for the tests that have failed.
• ERROR - You can use this option to publish test results for the tests that included errors.
• INCONCLUSIVE - You can use this option to publish test results for the tests that were inconclusive.

The values specified here override the values selected in Window > Preferences > Test > DevOps Test Hub > Results of Test UI.

You must prefix with ! to publish all the reports except the specified one.

For example, -publishreports "! TESTLOG"

All the reports except the TESTLOG report is published to DevOps Test Hub after executing the command.

For example, -results folder/resultname

After you run a test from the command line, the following outputs are displayed to give you the overall information of the test:

• --VERDICT: The verdict of the test.
• --REMOTE_RESULT: The URL of the result published to DevOps Test Hub.
• --REMOTE_RESULT_UI: The URL of the result published to DevOps Test Hub and can be opened in a browser to analyze the result.
• --LOCAL_RESULT: The path of the result saved locally.

For example, -workspace workspace_full_path -project proj_rel_path -publishpublish_url -stdout

You must ensure that both original and new datasets are in the same workspace and have the same column names. You must also include the path to the dataset when you run the -swapdatasets command.

For example, -swapdatsets /project_name/ds_path/ds_filename.csv:/project_name/ds_path/new_ds_filename.csv

You can swap multiple datasets that are saved in a different project by adding multiple paths to the dataset separated by a semicolon.

-swapdatsets /project_name1/ds_path/ds_filename.csv:/project_name1/ds_path/new_ds_filename.csv;
/project_name2/ds_path/ds_filename.csv:/project_name2/ds_path/new_ds_filename.csv

• compound - selects all the compound tests in the project for test run.
• test - selects all the Web UI, mobile, HTTP, and Windows tests in the project for test run.

To run a Web UI test on a different browser than that was used for the recording, specify the predefined variable. For more information, see Defining a variable to run a test with a selected browser.

For example, when you use-vmargs -Xmx4096m, specify a maximum heap size of 4096m. This method is similar to specifying -Xmx4096m in the eclipse.ini file for the workbench when playing back the test from the user interface.

To collect the response time data for the app itself and for the server and network and display them in different bar charts, use -vmargs "-De2e.collect=true". For desktop-based web applications, the response time data is captured and displayed by default.

To execute tests in parallel on all mobile devices, which are in passive mode, connected to the workbench and ready for playback, use -vmargs "-Dall.available.targets.in.parallel=true".

To execute tests in parallel on all supported desktop browsers and connected mobile devices, use -vmargs "-Dall.available.targets.in.parallel=all".

To execute tests in parallel on selected desktop browsers and connected mobile devices, use -vmargs "-Dall.available.targets.in.parallel=browser1,browswer2,browser3". You must separate browser names with a comma. For example, firefox, ff, chrome, ie, ie64, safari, "-Dall.available.targets.in.parallel=browser1,browser2,browser3".

You can use this option to customize the appearance of the PDF files, before you export the unified reports. The arguments and values are described as follows:

• To highlight only the verification points in test steps and exclude the tabular form of the step details in the PDF, you must use -vmargs "-Dreport.skipTableInPdf=true".

  To view details of test steps in the tabular form without highlighting the verification points in the PDF, you must use -vmargs "-Dreport.skipTableInPdf=true". This setting is the default preference for the exported PDF files for unified reports in HCL DevOps Test UI.

• To increase the size of screen captures that are displayed in the PDF, you must use -vmargs "-Dreport.snapShotSize=Size of images relative to the page-width". You can enter a value between 50 and 100 so that the images are increased to fit within the page margins in the exported PDF.

  For example, you must enter the following parameter to increase the width of the images to fit within 75% of the page width while the height gets increased proportionately: -Dreport.snapShotSize=75

  Note:

  • You can use this parameter, only when you have set -Dreport.skipTableInPdf=true.
  • If the parameter value is empty, HCL DevOps Test UI automatically sets the size of images to 50% of the page width as the default size.

• To exclude screen captures of specific SAP GUI test steps in the PDF, you must use -vmargs "-Dreport.skipTableInPdf=objectType1:methodName1|objectType2:methodName2". You can use the pipe character | as the delimiter between different objects.

  For example, you must enter the following values for the parameter to exclude images captured for the maximize and setTest methods of the GuiMainWindow and GuiCheckBox objects, respectively: -Dreport.skipActionEventImageList=GuiMainWindow:maximize|GuiCheckBox:setTest

  Note:

  • This parameter is applicable for SAP GUI applications only.
  • You can use this parameter only when you set -Dreport.skipTableInPdf=true.

For more information, see Customizing PDF files exported from unified reports.