Collecting diagnostic information after a server or client crash

The automatic diagnostic collection tool collects diagnostic data after server and client crashes and sends the collected data to a mail-in database when the server or client restarts. You can then use the collected data to determine the cause of the crash.

Data is stored in Fault Report documents. One mail-in database in each domain. You can store all of the Fault Report documents from all client and server crashes in a domain in one database, or you can create one database for server crashes and one database for client. You specify the mail-in database when you set up this feature. The collected data from client crashes can be sent to the customer's local Automatic Data Collection database(s) enabling customers to quickly identify duplicate problems encountered at their site, or the data can be sent to the same database as the server data.

Fault Analyzer is a server add-in task that processes all new crashes as they are delivered to the Automatic Data Collection mail-in database. For each new crash, the fault analyzer task searches the database containing the Fault Report documents and determines whether the stack matches a crash that has already been seen by a user or server at that customer site. The Automatic Data Collection database lists all Fault Reports, as well as Response documents for any duplicate occurrences of the same crash and indicates whether the duplicate occurrence is an exact match or a partial match of the original crash. The duplicate occurrences response documents are the Exact Match Fault Report and the Partial Match Fault Report documents. The Partial Match Fault Report document also includes a "percentage match" that indicates the percentage of the report that matches the original Fault Report for the crash.

Use the desktop policy settings document, Diagnostics tab, to set up automatic diagnostic data collection for clients.

Use the fields on the Diagnostics tab of the Server Configuration Settings document to specify whether the Fault Analyzer task is enabled on a server. When fault analyzer is enabled, at server startup, Domino® reads the Server Configuration Settings documents and desktop policy documents in the local Domino® Directory. If any document has a Fault Reports database specified, Domino® determines whether the database resides on the local server and if so, Domino® adds it to its list of databases to monitor. Every 10 seconds the process determines whether the data modified time of any of the its monitored databases has changed and if so, fault analyzer scans for new unprocessed documents to try to match.

An occurrence count and unique ID count are updated in the parent Fault Report document for the crash. The occurrence count is the total number of times a crash has occurred; the unique ID count is the number of clients and servers on which that problem has been reported.

The Administrator section of the Fault Report document contains a Resolved field that you can use to mark a crash as resolved for all versions of clients and servers reporting to the database, or resolved for specific versions that you identify by release level, for example, Domino® 6.5.1 or by hot fix numbers, for example, 652HF10.

When a fault is marked as resolved, all clients and servers that crashed with the same signature and that are at a version level for a fault that is to be resolved are not be marked as a duplicate. Instead, the Fault Report document is kept as a parent document. Resolved documents have a check mark displayed next to them in the views.

The following files that were previously stored in the Domino® data directory are now stored in the diagnostics directory, IBM_TECHNICAL_SUPPORT, located beneath the Notes/Domino data directory:

  • NSD output
  • memcheck output
  • core files
  • memory dump files (created in the format memory_platform_machine name_date@time.dmp instead of memory.dmp)
  • notes_child_pid

To prevent the diagnostic files from becoming quite large and using a significant amount of disk space, you can specify the number of days that these files can be stored before they are deleted by the automatic diagnostic collection tool.

When setting up automatic diagnostic collection for clients, you can designate whether this feature is invisible to users or whether users should be prompted to designate whether they want to send a diagnostic report to the mail-in database. If you designate that the user is not to be prompted, this feature is transparent to the user. The process then runs in the background after the user restarts the client and enters their password. If you designate that the user should be prompted to specify whether to report the crash, the user can also designate whether to view the report before it is sent.