Adding an LLM Model document

Use configuration in dominoiq.nsf on the Domino IQ administration server to download pre-trained data models in GGUF format.

Before you begin

Only models in GGUF format can be used on the Domino IQ server.

About this task

You can also choose to manually download an LLM model in GGUF format to the llm_models folder in the Domino data directory on the Domino IQ server. For more information see Downloading an LLM model manually.

Before the Domino IQ server downloads a model that is configured for download in an LLM Model document, the IQ server checks for model availability in the llm_models folder periodically (every 5 minutes). If the model doesn't exist, the Domino IQ task starts downloading the model from the specified download URL.
Note: Domino IQ also has the ability to copy a model. The download ability is limited to the administration server of the DominoIQ database, and that server can download models that it itself does not run. If a model is configured to run on a non-administation server, that server will check to see if the model exists locally. If it does not exist and the model status indicates that it has been downloaded to the administration server, the server will pull a copy of the model from the administration server to the current server. As a security restriction, any server that will be pulling models from the administration server must be listed in the administration servers Server document as a Trusted server under the Security tab. Models must be located under the llm_models subdirectory on the administration server.

Procedure

  1. From the Domino Administrator client, open the Domino IQ database, dominoiq.nsf, on the Domino IQ Administration server.
  2. From the Models view, click Add Model.
  3. Add the name and description for the model that you want downloaded.
  4. In the Model Download field, select Enabled.
  5. In the Download URL field, specify the URL to download the model from.
  6. (Optional) In the SHA2 Hash field, you can specify the SHA2 hash to verify the hash of the download model file.
    Note: If the SHA2 hash doesn't match, the model file is not saved to the llm_models folder and status field in the Model document is marked as failed. You can edit a failed download in the Model document and download status is reset if the hash value is corrected.
    The following screenshot shows an example of a download URL and SHA2 hash for downloading pre-trained models in GGUF format from opensource Ollama's LLM registry.

  7. Save the Model document.
  8. LLM Model download over HTTPS can also be optionally configured using a authenticated HTTPS proxy like squid or NGINX proxy. Use the Global Settings button in the DominoIQ database.


What to do next

Adding a Domino IQ Configuration document