Test creation with Test Genie

Test Genie is an AI-powered capability in HCL DevOps Test Hub (Test Hub) that automatically translates your natural language instructions into usable test scripts for web applications.

Traditionally, creating automated tests requires manually recording interactions against a live application or writing a test code. Test Genie accelerates this process so that you can define test steps by using plain text prompts. You provide a descriptive prompt, such as adding specific items to a shopping cart and proceeding to checkout, and the AI generates the corresponding test assets.

To improve the accuracy of the generated test steps, Test Genie uses the application context. When you initiate a generation request, the system passes your natural language steps along with the application URL and any application descriptions to the AI model. This context helps the AI understand the target interface and correctly map your instructions to the underlying web elements.

Supported AI providers

Disclaimer:

This release supports Anthropic in HCL DevOps Test Hub (Test Hub) as a Tech Preview. The Tech Preview is intended for you to view the capability of Anthropic for test generation offered by HCL Test Hub, and to provide your feedback to the product team. You are permitted to use the information only for evaluation purposes and not for use in a production environment. HCL provides the information without obligation of support and "as is" without warranty of any kind.

To use Test Genie, you must configure at least one AI provider in the Infrastructure > Cloud Credentials page of Test Hub. Test Genie supports the following AI providers:
  • Anthropic - Tech Preview
  • OpenAI
  • GitHub Copilot
    Restriction:
    Only one test generation session through GitHub Copilot can be active at a time across the entire Test Hub server. Any additional Copilot requests are queued and wait until the current session completes.
For optimal results, you can use OpenAI with the GPT-5.4-mini model.

Capabilities and limitations

Test Genie streamlines the initial creation of test scripts, but it operates within specific boundaries regarding test validation and test modification. The following points describe the scope of Test Genie:

  • Generative scope: Test Genie generates the navigational and transactional steps of a test, such as button clicks, text entry action, select actions from drop-down controls, drag-and-drop interactions, and navigation between pages. It also provides test generation details, including the model used and token usage metrics.
  • Iterative modifications: You can use AI prompts to modify or append steps to a test that Test Genie has already generated. To modify the AI-generated steps, you must submit a revised prompt and then generate the test again.
  • Verification points: The AI does not automatically generate or insert verification points into the test script. You must provide prompts to add verification points at specific points if you want to validate application behavior and data.
  • Datasets: Test Genie does not create datasets or automatically link existing dataset files to the generated test. You must configure datasets manually.

  • Error handling and messages: Test Genie provides clear error messages for troubleshooting during execution and creation:

    • Missing AI provider: Displays a validation message when you do not have an AI provider (such as OpenAI or Anthropic) configured in the cloud credentials settings.
    • Missing application URL: Prompts when the target application URL context is omitted.
    • Generation failures and stop reasons: Provides informative error messages and clear stop reasons when Test Genie cannot navigate the application, cannot complete the next intended action, or when test generation fails or becomes incomplete. Test Genie can indicate failures that are caused by timeouts, invalid page structures, provider service interruptions, and provider responses such as 4xx client errors or 5xx server errors. These details help you identify whether you must refine the prompt, update the application context, modify the generated steps, or retry the request.

Session limits for AI generation

In Test Hub, you can configure session limits as Advanced configuration values when you add a cloud credential for an AI provider in Cloud Credentials. These values are saved with the selected provider credential and are applied whenever Test Genie starts a suite or test generation session by using that credential.

The three limits are Max tokens per session, Max interactions per session, and Max duration per session (in seconds). During test generation, Test Hub evaluates all three limits. If any configured limit is reached, the active AI conversation session ends, and you must start a new session to continue the test generation.

These session limits are optional for OpenAI and Anthropic credentials and are not applicable for GitHub Copilot credentials. By configuring these values, you can control response depth and generation time with platform resource usage for large or complex suite-generation workloads.