Performance testing tips

Use these tips to make HCL DevOps Test Performance (Test Performance) run faster and more efficiently.

The following suggestions can help you to get the best performance from Test Performance:

  • Number of computers. Have at least two computers for a test. The user interface consumes significant resources; therefore play back a test or schedule on a computer (agent) that is separate from the computer that is running the workbench (UI).
  • Number of virtual users at remote locations. When you assign a user group to a remote location, do not overload the remote computer (agent). If you exceed the number of virtual users that the remote computer can run, the performance measurements of the server will be skewed because they will be affected by the performance of the computer. The test results will reflect the load of the computer more than the load of the server. For best results on a computer with a 1 GHz processor and 1 GB of RAM, do not exceed 1000 concurrent virtual users.
  • TCP/IP ports. Your computer must have a sufficient number of TCP/IP ports. On computers with Microsoft™ Windows™, the typical limit is 5000. Issue the netstat -a command to observe port use. If the largest number you see is 5000, then you need to increase the number. To increase it, open the registry. Under HKEY_LOCAL_MACHINE/SYSTEM/CurrentControlSet/Services/Tcpip/Parameters, create a new dWord named MaxUserPort, and set its value up to 65000. Restart the computer.
  • Open file limit for Linux™. Computers that are running Linux™ need a per-process open file limit higher than 1024. As root, enter ulimit -n 30000 (or another appropriate value) before starting Agent Controller.
  • Looping within tests. If you are stress testing a server, your test typically contains a loop. Your connection behavior differs depending upon whether the loop is set at the schedule level or at the test level. Setting a loop at the test, rather than the schedule, level gives you a performance advantage, because the connections are reused during the looping process. For more information, see Add a loop.
  • Logging levels. After the test is stable, for maximum performance, reduce the test log level and problem determination log level and sample a small number of users. Increase the statistics sample interval to 30 or 60 seconds for long-running tests.
  • Workbench heap size. The Java™ Virtual Machine (JVM) heap size on the workbench is based on the available physical memory. Do not run the workbench on a computer with less than 768 MB of physical memory. The maximum workbench heap size depends on your JVM. Although the heap size is not strictly necessary for playback performance, you can increase the workbench heap size. To increase the heap size, set the -Xmx parameter in the eclipse.ini file, which is located in the product installation directory. For Windows™, if your physical memory is 3 GB or more, then the maximum heap size must not exceed 1200 MB. For Linux™, the maximum heap size is approximately 3000 MB. If the workbench is sluggish or fails to start after you increase the heap size, reset the heap size to the default by removing the VMARGS=-Xmx line from the eclipse.ini file.
  • Location (agent) heap size. To access maximum heap, after one successful test of any size, search for a location (agent) attribute called RPT_DEFAULT_MEMORY_SIZE. If you cannot find this attribute, you can specify a maximum heap by creating a new attribute: RPT_VMARGS=-Xmx1500m (for example, max heap 1.5 GB). For more information, see Increasing memory allocation.
    Test Performance sets heap size for RPT_DEFAULT_MEMORY_SIZE based on the bit-type of the JRE:
    • For 32-bit Java Runtime Environment (JREs), Test Performance sets 70% of the size of physical memory to RPT_DEFAULT_MEMORY_SIZE. Typically, the maximum limit is set to 1200m.
    • For 64-bit JREs, some workloads might perform better with a lesser heap size than 70% of physical memory up to a maximum of 12000m.
  • Disk space. Verify that there is sufficient free disk space on the workbench and agent computers. Also, verify that there is sufficient free disk space on the drive that contains the system temporary directory.
  • Recording length. If you record for a relatively long time, test generation also takes a long time. If test generation is taking a relatively long time, try shorter recording scenarios.