CompletionStream method (LLMReq - Java)

Sends a chat completion request to a large language model (LLM) hosted by the Domino IQ server and receives a response in Stream mode.

Defined in

LLMReq

Syntax

public void completionStream(String server, String command, String userPrompt, CompletionStreamCallback callback) throws NotesException

Parameters

String server

The Domino Server IQ name that is needed to send the request. If empty, the request will be routed to the Domino IQ server configured to service the command. For more information on Domino IQ server configuration, see Enabling Domino IQ servers.

String command

The name of a command document defined in the Domino IQ configuration database. The command document specifies a system prompt that guides the LLM behavior and responses.

String userPrompt

The user prompt is the specific input given by the caller of this method. This prompt is merged with the system prompt defined in the command document and then presented to the language model through the Domino IQ server.

CompletionStreamCallback

Interface with one method name callback which is executed when receive one response from DominoIQ server. Here is the implementation of the callback interface:
interface CompletionStreamCallback {
CompletionStreamAction callback(boolean LastResponse, String Content);
}
The callback method has two parameters:
  • boolean LastResponse: Receive value true if it is the last response from stream
  • String content: next chunk of the content of the response from the stream
The callback return enum object CompletionSteamAction has two values: Continue or Stop. When return Continue the Stream will continue to next response, if return Stop stream will cancel and stop response.
With Java ‘s lambda functionality, You would be able to use the call back interface like this:
llmreq.completion(server, command, prompt, (lastCompletion, content) -> {
    // Do work with the content
    return CompletionAction.Continue;
});

Example

import lotus.domino.*;
import lotus.domino.LLMReq.*;
public class JavaAgent extends AgentBase {	
    public void NotesMain() {
      try {
	      Session session = getSession();
             AgentContext agentContext = session.getAgentContext();
             String command = "sampleCommand"; // Should replace to your command which configure in Domino Configure Database
             String userPrompt = "sample user prompt" // Should replace to your real user prompt to send to DominoIQ server
             LLMReq llmreq = session.createLLMRequest();
          if (llmreq.isCommandAvailable("Test" , cmd))
          {
                // Send to Domino IQ server name Test  
 LLMRes llmres = llmreq.completion("Test", cmd, userPrompt); 
        	  if (llmres.getFinishReason() == LLMRes.FINISH_REASON_STOP)
            	  System.out.print("Content: " + llmres.getContent());
		else
			{
			System.out.println(content);
			System.out.println("************ last response ****************");
			}
		return CompletionStreamAction.Continue; // If you want to stop the stream, use return CompletionStreamAction.Stop;
	        	});
          else
        	  System.out.println(command + " Command is not available");

      } catch(Exception e) {
          e.printStackTrace();
       }
   }
}