CompletionStream method (LLMReq - Java)

Sends a chat completion request to a large language model (LLM) hosted by the Domino IQ server and receive response in Stream mode.

Defined in

LLMReq

Syntax

public void completionStream(String server, String command, String userPrompt, CompletionStreamCallback callback) throws NotesException

Parameters

String server

The Domino Server IQ name that is needed to send the request. If empty, the request will be routed to the Domino IQ server configured to service the command.

String command

The name of a command document defined in the Domino IQ configuration database. The command document specifies a system prompt that guides the LLM behavior and responses.

String userPrompt

The user prompt is the specific input given by the caller of this method. This prompt is merged with the system prompt defined in the command document and then presented to the language model through the Domino IQ server.

CompletiongStreamCallback

Interface with one method name callback which is executed when receive one response from DominoIQ server. Here is the implementation of the callback interface:
interface CompletionStreamCallback {
CompletionStreamAction callback(boolean LastResponse, String Content);
}
The callback method has two parameters:
  • boolean LastResponse: Receive value true if it is the last response from stream
  • String content: is content of the response from stream
The callback return enum object CompletionSteamAction has two values: Continue or Stop. When return Continue the Stream will continue to next response, if return Stop stream will cancel and stop response.
With Java ‘s lambda functionality, You would be able to use the call back interface like this:
llmreq.completion(server, command, prompt, (lastCompletion, content) -> {
    // Do work with the content
    return CompletionAction.Continue;
});

Example

import lotus.domino.*;
import lotus.domino.LLMReq.*;
public class JavaAgent extends AgentBase {	
 public void NotesMain() {
      try {
	Session session = getSession();
         	AgentContext agentContext = session.getAgentContext();
          	String command = "sampleCommand"; // Should replace to your command which configure in Domino Configure Database
              String userPrompt = "sample user prompt" // Should replace to your real user prompt to send to DominoIQ server
           LLMReq llmreq = session.createLLMRequest();
          if (llmreq.isCommandAvailable("Test" , command))
                   // Send request to DominoIQ server name Test
llmreq.completionStream("Test", command,userPrompt, (lastResponse, content) -> {
	        	    // Do work with the content for example print content to console
	        	if (!lastResponse)
			System.out.println(content);
		else
			{
			System.out.println(content);
			System.out.println("************ last response ****************");
			}
		return CompletionStreamAction.Continue; // If you do not want to stop the stream use return CompletionStreamAction.Stop;
	        	});
          else
        	  System.out.println(command + " Command is not avaivale");


      } catch(Exception e) {
          e.printStackTrace();
       }
   }
}