How synced data moves through the system

Device registration

When a device needs to sync data with the user's mail server, the user registers that device with the HCL Traveler server. This involves some initial setup which includes negotiating security settings as well as data options and filters to be used when syncing the data from the user's mail database to the device and vice versa. After these initial steps, the device is eligible to sync data. As a result, the HCL Traveler server starts monitoring the user's mail database for data that could be synced to the device.

Monitoring the mail file for changes

After an initial determination, the HCL Traveler server watches for any changes to the user's mail database since the last time it was checked. This is done using a Domino API call which returns the databases being monitored that have changed since the given time. The interval in seconds between checks of the mail server for changes is configured in the server document for the Traveler server (Monitor Polling Interval). This can also be overridden with the notes.ini setting NTS_PUSH_MONITOR_INTERVAL. Default is 3 seconds. All communication between the HCL Traveler and mail servers is done through NRPC (default port 1352).

Because a user can have mail database replicas on many servers, HCL Traveler functions as a "Master Monitor" which is responsible for determining which mail server's replica should be monitored. At any given point, only one mail server is being monitored for a user. The Master Monitor is responsible for which server is being monitored, switching servers if the current server goes down, and trying to switch back to the primary server whenever it is not the current server. If the HCL Traveler server switches to a secondary mail server due to a failure accessing the primary, the server will continue to monitor the secondary mail server for a configurable amount of time (notes.ini NTS_PUSH_MONITOR_PRIMARY_SERVER_RETRY_TIME) before re-trying the primary. The default interval before re-checking the primary is 60 minutes. In the event that there are no mail servers available for the user, the server will wait a configurable interval (notes.ini NTS_PUSH_MONITOR_SERVER_RETRY_TIME_INTERVAL) before retrying the the user's mail servers (primary first). This default retry interval is 15 minutes. The HCL Traveler server monitors the set of databases from each mail server on one thread per server; these threads are named starting with "Mntr". If the server is part of a High Availability (HA) pool, only one HCL Traveler server in the pool is the Master Monitor for the user (and all the user's devices).

Processing the mail changes (Prime Sync)

Once the HCL Traveler server detects that a mail database has changed, it creates a new thread to perform a Prime Sync (PS) to evaluate the changes; these threads are named starting with "PS". Initially, this involves looking through all of the data in the user's mail database to determine what should be synced to the device (based on the data options and filters established previously) and what should not be synced to the device.

After the initial prime sync, the prime sync only needs to evaluate any items that have changed since the last time the mail database was checked. For each item that has changed, the HCL Traveler server determines if it needs to be synced to the device or not. This determination is then stored in the HCL Traveler DB (only the meta data about the document - not the entire document) for when the device requests the information.

Notify the client of changes (Push)

As the device always initiates the device sync requests, the server needs a way to tell the device to initiate a device sync. This is what is referred to as "push", but really it is just the server telling the device to make the device sync request as the server has new data for the device. The HCL Traveler server prime sync process attempts to send a push message to the device if data is determined to be needing to be synced to the device. The push message from the Traveler server can be sent using HTTP (the device makes an HTTP request with a long timeout such that the HCL Traveler server can hold the request to respond to when changes are available or when the timeout is reached), FCM (Firebased Cloud Messaging), or APNS (Apple Push Notification Service).

The end to end process of push notifications triggering the device to sync the changes is known as "auto-sync". See Setting auto sync options for information on enabling automatic synchronization.

Device synchronization

At this point, the HCL Traveler server knows what data needs to be synced to the device once the device asks for the data. The process of the device asking for the data is called the Device Sync (DS). The device sync is always initiated by the device, which makes an HTTP request (HTTP defaults to port 80, HTTPS defaults to port 443) to the server with parameters and HTTP body that is either SyncML or Microsoft Exchange ActiveSync (the two sync protocols that HCL Traveler supports).

The HTTP request will be made to the server seen in the settings on the device, but it should be the same server as the HCL Traveler server has configured through the External URL. This HTTP request will go through any HTTP load balancers, proxies, firewalls, and network appliances until it reaches the Domino HTTP server where the HCL Traveler process is running.

When the HTTP request reaches the Domino HTTP server, Domino HTTP will do the HTTP authentication. This can be done through any of the normal HTTP authentication mechanisms, including names.nsf or Directory Assistance. If the request is successful, it is passed to the HCL Traveler servlet running inside the Domino HTTP process.

The servlet takes the HTTP request data, wraps it in a serialized object, sends the serialized object to the HCL Traveler process running on the same server (always the same server) using a TCP socket (default port 50125), and waits for a response from the HCL Traveler process. While the servlet is waiting for the response, the servlet is holding the HTTP thread as the thread must be held until the response is ready.

The HCL Traveler process receives the serialized object and queues it for a Worker thread to be assigned to it; these threads are named starting with "Worker". As the HTTP process already completed the authentication, the HCL Traveler server does not perform any authentication - the user is whomever HTTP identified the user as. All incoming requests (including but not limited to device syncs), with the exception for tell commands, are serviced through a Worker thread.

If the server is part of a High Availability (HA) pool, then the request is load balanced as needed . If load balancing is needed, the same serialized object is sent using a TCP socket (default port 50125) to the other HCL Traveler server which is the Master Monitor for that user, and the receiving HCL Traveler process puts the request on its own Worker thread and continues.

As a device sync using the SyncML protocol may span multiple HTTP requests and responses, the Worker thread creates a new DS thread to cache sync state across the multiple HTTP requests and responses; each incoming request will be serviced by a new Worker thread, but they will all be placed on the DS thread for that device until the sync is completed. At that point, the DS thread is recycled. Even if the device sync will be handled in one HTTP request and response (Microsoft Exchange ActiveSync is always only once), it is still placed on a DS thread for consistency. The DS thread looks at the meta data stored in the HCL Traveler database from the prime sync process, retrieves the data from the user's mail database, and builds the HTTP response.

When the DS (only for device syncs) or Worker (the other requests) thread has built the HTTP response, the thread takes the HTTP response data, wraps it in a serialized object, and sends the serialized object to the servlet which is holding the HTTP thread using a TCP socket (default port 50126). If the request was not load balanced, this is the HTTP process running on the same server as the HCL Traveler process. If the request was load balanced, this would be the servlet on the original HTTP/Traveler server; note that the response is not going back through the original HCL Traveler process - it goes directly back to the servlet. The servlet matches up the response to the request and the HTTP response is sent back through the network to the device.