Working with PeerNest
The BigFix client includes a new feature named PeerNest, that allows to share binary files among clients located in the same subnet. The feature is available starting from BigFix Version 9.5 Patch 11.
A practical use case is a branch office connected to the data center through a slow link: with earlier BigFix versions, the suggested configuration required a Relay to be installed in the branch office in order to download and cache large payloads.
With PeerNest, the BigFix Clients can share downloaded binaries and therefore reduce the number of communications going outside of the branch office even if a Relay is not installed locally. In this way, multiple Clients generate on the Relay the download load of a single Client, because only one Client downloads from the Relay and then shares the download with the peers.
Use of PeerNest can help reducing the number of Relays in some complex BigFix deployment scenarios, therefore reducing infrastructural costs.
An introduction video can be found here: https://www.youtube.com/watch?v=tXRX3zlw1aQ.
Enabling PeerNest
To enable the PeerNest feature, set to 1 the following configuration setting on the Client:
_BESClient_PeerNest_Enabled = 1
The Client enables all the PeerNest feature in order to locally optimize the download of binaries.
This configuration setting requires a restart of the Client to be effective.
PeerNest configuration settings
For details about all available settings, see Peer to peer mode.
PeerNest in depth
When multiple Clients are executing actions requiring the prefetch of a binary file, they check with their peers if the file is already cached in the subnet. If the binary was not cached then the Clients can elect one of them as responsible for downloading from the Relay (Figure 1): the peer with the highest priority, among the peers requiring the file, will manage the download; if all peers have the same priority, the computer with the lowest ID will download the file from the Relay. By doing so, for a given action, all files will likely be downloaded from the Relay by the same computer, and so the downloads will be serialized (one file at a time), thus ensuring minimum bandwidth occupation in the link between Relay and computer.
- A will be elected as master as it has the highest priority among the peers requiring the file (A, B, E).
- C and D do not require the file, so even if D has higher priority than A, D is not involved in the master election process.
Figure 1: Master election
As soon as the elected master (A) starts downloading the file from the Relay (Figure 2) it notifies the other peers that there is a download in progress, so they wait for it without taking further actions. The master sends periodic notifications about the download.
Figure 2: Download in progress
If the peers did not receive the download in progress message (download failed, Client down, network issue), a new election process is started over and another peer becomes master.
When the elected master finishes the download, it moves the file into the PeerNest cache and notifies the other peers about its availability; the peers interested in the file then start downloading it from master and share it, using the same mechanism (Figure 3). In this way, multiple Clients generate on the Relay the download load of a single Client, because only one Client downloads from the Relay and then shares the download with the peers.
The following figure shows how the peers interested in the file (B, E) start downloading it directly from peer A, instead of downloading it from the Relay. When their download completes, the file is cached to be available for future usage: so the Clients A, B and E will share the downloaded file with C and D.
Figure 3: File sharing
Clients with peer enabled will start up an HTTP Server listening on port 52311 for peer
transfers. Each Client can serve at the same time a maximum of
_BESClient_PeerNest_MaxActiveFileDownloads
other peers (default value is 5).
_BESClient_Download_RetryMinutes
and
_BESClient_Download_RetryLimit
Client settings – with the following addition: - The peer from which a failed download attempt has occurred gets removed from the memory peer list (unless that peer is the only one in memory), and so the next attempt will be done with another peer.
- If the retry-count reaches the limit, the Client will go to download the file directly from the Relay.
If the Client fails to download the file because the peer that it has tried to connect to is already serving 5 downloads (default case), the retry-count is not increased and the peer is not removed from the memory peer list.
Best Practices
A good practice is to assign higher priorities to computers with a better link to the Relay and enough resources to serve the other peers of the subnet (and possibly a steady power supply).
PeerNest requires an increased disk storage space for caching files. The default PeerNest
cache size is 2GB, which is enough for many scenarios; it must be increased in case of transfer
of large files (patch management, software distribution, etc.) in order to fit the cache. The
PeerNest cache is intended as a temporary storage, so usage and lifetime can be fine tuned with
the following parameters: _BESClient_PeerNest_DownloadsCacheLimitMB
,
_BESClient_PeerNest_MinimumDiskFreeMB
,
_BESClient_PeerNest_MinimumCacheDays
,
_BESClient_PeerNest_MaximumCacheDays
.
PeerNest requires UDP (port 52311) communication to be enabled, in order to allow the BigFix Clients to communicate with each other.
It also requires TCP (port 52311) to allow the BigFix Clients to download files from a peer, and the subnet supporting multicasting.
It is recommended to set PeerNest in passive mode (using the
_BESClient_PeerNest_IsPassive
configuration setting) on Clients that cannot
open this port or do not want to use additional disk space for caching. Passive Clients will
only download from the other peers but will not share content. The following figure shows an
example of configuration.
Bandwidth Throttling
Clients with peer enabled will start up an HTTP Server; the other peers can connect to it for downloading.
Each Client serving files to the other peers can control the amount of bandwidth allocated for
that purpose. The _BESClient_HTTPServer_ThrottleKBPS
setting defines the total
number of kilobytes that the Client gives to all of the peers combined per second (0 means no
limit): if its value is 1000 KB/sec and there are 10 peers downloading simultaneously, the
Client will send data to each at 100 KB/sec (for a total of 1000 KB/sec).
Troubleshooting scenario 1
On BigFix Clients hosted on an Operating System that has the Internet Protocol version 6 (IPv6) disabled or not configured:
If you want to use the PeerNest feature, you must:
- Set on these Clients the
_BESClient_Comm_IPCommunicationsMode
configuration setting as follows:_BESClient_Comm_IPCommunicationsMode = OnlyIpv4
- Restart the Clients for the changes to take effect.
Troubleshooting scenario 2
On BigFix Clients that have an
active polling set using the _BESClient_Comm_CommandPollEnable
and
_BESClient_Comm_CommandPollIntervalSeconds
configuration settings:
If you want to use the PeerNest feature, you must not configure these Clients to be "passive"
PeerNest agents. Do not enable on them the _BESClient_PeerNest_IsPassive
configuration setting. Otherwise, depending on the timing of the polling, multiple Clients in a
subnet can download the same binary, without sharing it.