Adding a Domino IQ Configuration for Remote mode

In Remote mode, every Domino IQ server needs its own Configuration document that associates the Domino server name with an LLM (model) used by the AI inferencing engine running on a the remote server, outside of the HCL Domino IQ server instance.

Procedure

  1. Using the Domino Administrator client, open the Domino IQ database , dominoiq.nsf, on the Domino IQ Administration server.
  2. Select the Configurations view and click the Add Configuration button.
  3. On the Basics tab, complete the following steps:
    1. In the AI endpoint field, select Remote.
    2. Select the name of a Domino IQ server from the list.
    3. Specify the name of the download model from the LLM Model document. Use the model naming convention that the remote AI server typically specifies for launching its inferencing engine. The model name is sent as part of the /v1/chat/completions request payload.
    4. Set the Status field to Enabled, so that the Domino IQ task gets loaded on the server.

      Sample Configuration document for Remote mode

  4. Provide the AI endpoint URL. The endpoint supported is HTTPS only. For example: https://endpoint-serv.example.com/v1/chat/completions
  5. Provide an API Key to use for the remote AI endpoint. This is the only form of authentication supported by HCL to the remote AI endpoint/server, as it secures the requests sent over TLS to the remote AI endpoint/server.

  6. Add the Trusted roots for the remote AI server to Domino's Certstore database. See Adding trusted root certificates for more information.

  7. If you're using a guard model, complete the following steps to enable it. Otherwise, skip to step 8.
    1. Click the Guard Model tab.
    2. Enter the name of the LLM guard model. Use the model naming convention that the remote AI server typically specifies for launching its inferencing engine. The model name is sent as part of the /v1/chat/completions request payload.
    3. Set the Guard model status field to Enabled.


    4. Select Check LLM responses if you want the Guard Model to validate both the user content sent to the LLM and the response content received from the LLM. If you only want to validate the user content, leave this option unchecked.
    5. Provide the AI endpoint URL. The endpoint supported is HTTPS only. For example: https://endpoint-serv.example.com/v1/chat/completions
    6. If the remote AI server is required to use an API key when sending a request, provide the API key.
    7. In the Safe token value field, enter the your guard model's predefined keyword for when the content passes safety check. As examples, with LLAMA Guard 3, "safe" indicates that the request content does not fall into any risk categories defined by the model. For ShieldGemma, "No" indicates that the requested content does not violate the policy that the model was trained to protect.
    8. In the Unsafe token value field, enter your guard model's keyword for when the content doesn't pass the safety check. With LLAMA Guard 3, you would enter "unsafe" or, for ShieldGemma, "Yes".
      Note: For remote AI endpoints, the Advanced tab in the Configuration document doesn't contain any settings that apply to this configuration.
  8. Save the document.

What to do next:

Adding an LLM System Prompt document