Troubleshooting tips
Here is some helpful information if you encounter a problem with Domino IQ.
- Set DEBUG_DOMIQ=1 in the Notes.ini file of each Domino IQ server. Additional logging can be found in Domino's console.log file for the Domino IQ task and NotesLLM Query. Searching the console.log for DominoIQ will find most of the debug, the exception being some lower levels of code.
- The Domino IQ server sends the output of the AI inference server to the dominoiq_server.log file in the IBM_TECHNICAL_SUPPORT directory. The dominoiq_server.log is unique to each run with the format of the name being: dominoiq_server_<servername>_<YY_MM_DD@HH_MM_SS>.log
- To log basic responses from the AI inferencing server, set the Notes.ini DEBUG_DOMINOIQ_LLMREQUEST=1 on the Domino IQ server.
-
To get detailed trace logging to and from the AI inferencing server, set the Notes.ini DEBUG_DOMINOIQ_LLMPAYLOAD=1 on the Domino IQ server. This will log the detailed connection/request/response tracing to a file prefixed with "domiqllm" under the IBM_TECHNICAL_SUPPORT directory.
- There are various individual logs for different parts of the IQ feature in
IBM_TECHNICAL_SUPPORT directory:
- Logging for the query model: dominoiq_server.log
- Logging for the guard model: dominoiq_guard.log
- Logging for the embedding model: dominoiq_embed.log
- Logging for the vector database: dominoiq_vector.log
- Similarly, the payload logging is broken out by function:
- Query thread logging: domiiqllmthr<thread id>_*.log
- Embedded thread logging: domiiqembthr<thread id>_*.log
- Vector database thread logging: domiiqvecthr<thread id>_*.log
- Guard thread logging: domiiqgrdthr<thread id>_*.log
- The file qdrant_config.yaml gets updated with the hostname and port from the
vector configuration in the account document. There are cases where
extraneous characters get added to the end of the file causing the vector
database process to see it is a bad configuration file and exit. This can be
seen by examining the dominoiq_vector.log and seeing an error to the effect
of "Error: simple key expect '<?>" at byte <number> line <number>
column <number>" in theqdrant_config.yaml.
To resolve, edit qdrant_config.yaml to remove any text after the line : "http_port: <number>" and save the file. Use the server console command "dominoiq start" to relaunch Domino IQ.