Customizing HTTPS for downloads

This page describes the usage of CA bundles in HTTPS downloads and how to customize them.

On this topic, the word “downloads” refers exclusively to the effect triggered by the action commands prefetch and download. Such commands might also be invoked by BigFix applications (such as BigFix Inventory) to download catalogs or other required data.

Site gathering and/or communication between BigFix components (for example, server – client communication) are not affected by this topic.

For more details about customizing HTTPS for gathering, refer to Customizing HTTPS for Gathering.

Overview

The main purposes of HTTPS protocol are authentication of the accessed website and protection of the privacy, as well as integrity of the exchanged data. This is obtained through the verification of the remote server certificate, using a trusted third party (Certification Authority), and the channel encryption. The check is performed during the handshake, and, if it fails, the connection is dropped. It protects the communication against malicious attacks.

Starting from Patch 8, to support the download from an HTTPS URL, it is necessary to provide appropriate trusted certificates to verify the remote server identity.

Public CA, Private CA and self-signed certificates

Majority of BigFix Content works without additional changes in Platform configuration. Customization is needed only in specific cases, depending on the type of certificate used:

  • Certificates signed by external / public CA: no action needed.
  • Certificates signed by internal / private CA: needs the certificate to be added. See Custom certificates.
  • Self-signed certificates: needs the certificate to be added. See Custom certificates.

Certificates shipped with BigFix

The curl website offers a prebuilt package that contains the same certificates that are included with Mozilla.

This package is copied in a certificate bundle named ca-bundle.crt and distributed in the BES Support site. The certificate bundle is searched in different locations, depending on the BigFix version:

  • In Patch 8, both the server/relay and the client load the CA bundle from the BES Support site directory under the BigFix client installation path.
    Note: If the BigFix server/relay is running without administration privileges, for example a Domain user on Windows AD, it will not be able to read the folder located under the client path: in that case, a custom CA bundle must be specified. See the section below named Custom certificates.
  • Starting from Patch 9, for the BigFix server/relay, the bundle is loaded from the last gathered BES Support folder under the BigFix server/relay directory. This folder is always accessible, regardless of the privileges accorded to the process. For example, on a UNIX platform the bundle path could be:
    /var/opt/BESServer/wwwrootbes/bfmirror/bfsites/enterprisemirror_<SiteID>_<SiteVersion>
    In case the ca-bundle.crt file is not present or cannot be read, the server/relay will fall back on the default client path, as for Patch 8. For what concerns the BigFix client, the bundle location is the same as Patch 8.

Notice that BigFix does not perform any active monitoring on certificates validity and/or their expiration dates. No warning is provided prior to certificate expiration. Using expired or invalid certificates will result in download failure.

Enabling HTTPS downloads from untrusted sites

Starting from Patch 8, the following settings can be used to allow or disallow the download from untrusted sites:
  • _BESRelay_Download_UntrustedSites (server/relay). It is a boolean setting. When it is set to 0 (default value), the downloads from untrusted sites are not allowed. When it is set to 1, the downloads from untrusted sites are allowed.
  • _BESClient_Download_UntrustedSites (client). As described above, but the setting works for the client.

Custom certificates

Starting from Patch 8, custom CA bundles (included self-signed certificates) can be set for the components: server, relay and client. The setting changes according to the patch version.

In Patch 8, the following parameters can be set:
  • _BESRelay_Download_CACertPath (server/relay). It is a string setting where you can declare the full path of a custom CA bundle. It overrides the default one.
  • _BESClient_Download_CACertPath (client). As described above, but the setting works for the client.
Note: These parameters are still recognized in Patch 9 and above, but they are considered as deprecated and should not be used. This is done for retro-compatibility purpose. Their behavior is slightly different, though: unlike Patch 8, the custom bundle will be appended to the other certificates, and will not override them. Versions from 10.0.9 and later are also provided with a specific parameter to ignore the default certificates (see below).

Default directory for custom certificates

In Patch 9, a predefined folder is introduced for all the BigFix components (server, relay and client), <StorageFolder>/TrustedDownloadCerts, that can be used to store custom certificates and/or bundles. All the .crt and .pem files contained in this folder are added to the default certificates when an HTTPS download is performed. For example, on a default Linux/UNIX installation, the folder full path for the client custom CA bundle is: /var/opt/BESClient/TrustedDownloadCerts, while the full path for the relay custom CA bundle is /var/opt/BESRelay/TrustedDownloadCerts.

Only files with extensions .pem and .crt are loaded, and they can contain both certificates bundles or single certificates. No check is performed on such files: if a file contains invalid data, it might cause a download failure.

The following is a sample task that creates a .crt file inside the <StorageFolder>/TrustedDownloadCerts. This sample can also be used as a template for understanding how to retrieve the <StorageFolder>/TrustedDownloadCerts directory for server, relays and clients.

<?xml version="1.0" encoding="UTF-8"?>
 <BES xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="BES.xsd">
 	<Task>
 		<Title>Add Download Certificate</Title>
 		<Description><![CDATA[Sample Task for creating and downloading a custom certificate.]]></Description>
 		<Relevance>if exists property "in proxy agent context" then ( not in proxy agent context ) else true</Relevance>
 		<Category></Category>
 		<Source>Internal</Source>
 		<SourceID></SourceID>
 		<SourceReleaseDate>2023-02-14</SourceReleaseDate>
 		<SourceSeverity></SourceSeverity>
 		<CVENames></CVENames>
 		<SANSID></SANSID>
 		<MIMEField>
 			<Name>x-fixlet-modification-time</Name>
 			<Value>Tue, 14 Feb 2023 15:04:24 +0000</Value>
 		</MIMEField>
 		<Domain>BESC</Domain>
 		<DefaultAction ID="Action1">
 			<Description>
 				<PreLink>Click </PreLink>
 				<Link>here</Link>
 				<PostLink> to deploy this action.</PostLink>
 			</Description>
 			<ActionScript MIMEType="application/x-Fixlet-Windows-Shell"><![CDATA[
 // This is a sample Task for reading the TrustedDownloadCerts folder and copying a certificate
 // This Task will add a certificate in the bundle.
 // The certificate will be added in the Server folder if the Server is installed, on the Relay folder if the relay is installed, in the Client folder otherwise  
 
 if {exists main gather service}
     if {windows of operating system}
 	    parameter "storageDir" = "{value "StoragePath" of key "HKEY_LOCAL_MACHINE\SOFTWARE\BigFix\Enterprise Server\GlobalOptions" of registry as string}" 
 	else
 	    parameter "storageDir" = "{key ("StoragePath") of section ("Software\BigFix\Enterprise Server\GlobalOptions") of file "/var/opt/BESServer/besserver.config"}"
 	endif
 elseif {exists relay service}
     if {windows of operating system}
 	    parameter "storageDir" = "{value "StoragePath" of key "HKEY_LOCAL_MACHINE\SOFTWARE\BigFix\Enterprise Server\GlobalOptions" of registry as string}" 
 	else
 	    parameter "storageDir" = "{key ("StoragePath") of section ("Software\BigFix\Enterprise Server\GlobalOptions") of file "/var/opt/BESRelay/besrelay.config"}"
 	endif
 else
     parameter "storageDir" = "{storage folder of client}"
 endif
 
 delete __appendfile 
 
 appendfile -----BEGIN CERTIFICATE-----
 appendfile MIIDhTCCAm2gAwIBAgIEFtnq9zANBgkqhkiG9w0BAQsFADBbMScwJQYDVQQDDB5SZWdlcnkgU2V
 appendfile Zi1TaWduZWQgQ2VydGlmaWNhdGUxIzAhBgNVBAoMGlJlZ2VyeSwgaHR0cHM6Ly9yZWdlcnkuY29
 appendfile MQswCQYDVQQGEwJVQTAgFw0yMzAyMjcwMDAwMDBaGA8yMTIzMDIyNzEwMDYyMFowRzETMBEGA1U
 appendfile AwwKbXlzaXRlLmNvbTEjMCEGA1UECgwaUmVnZXJ5LCBodHRwczovL3JlZ2VyeS5jb20xCzAJBgN
 appendfile BAYTAlVBMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAh8F7GgL4IQrc8E4AHJBmdYu
 appendfile X7DEJ0X5XnXeD+tj5XVe7AE3lsoFqLkrNo/mWzZrp9y6ZHoew9AaLrJgUmFdU5c/1aUhWlVtBU+
 appendfile tOBqnZNRKEy+AsObZ2lYryNTVwAbTieTLVPM4pDNr6dwZen1tWfVIKg/2NiApTjwCdamalljP2J
 appendfile V+NxLsjFhGg8q8lD0JJiqoNUm1bi22JrqWJ8Z9cXXmuiNHzUxzVW7XaxzXyWXE4WNWZMIO/gUl5
 appendfile mP9VjXvyXymzMoeDxjMg9cyyrni5ZJsv8Uqkq6ni/Tml6QOYahjNOYmHF8j9v+vTLI/YwfFF4lU
 appendfile QXcOXsHxWyepkwIDAQABo2MwYTAPBgNVHRMBAf8EBTADAQH/MA4GA1UdDwEB/wQEAwIBhjAdBgN
 appendfile HQ4EFgQUeokEssftpr4W/DjX8EDSYkr9V3AwHwYDVR0jBBgwFoAUeokEssftpr4W/DjX8EDSYkr
 appendfile V3AwDQYJKoZIhvcNAQELBQADggEBAE64aGOnSfA0ePiu6CguxH5GKojTM6N7z2ouzPJGymrVPdM
 appendfile jNfb81Xm+VJ0Eervw8uz/D3NYVCJy4/Q+gvAyYGHFmRxmIue/4RjsOK482WkeICGTsZWggoz3wX
 appendfile 12ZX13noMCULcU1lI83eNBn3LFvA/bYH7cwSBGxmLidg1jkbATktZMQYJXIQ6cNtst4NgfMZPzj
 appendfile w6b9+1tf6ohk+zjYNWPf3N58mziQ3JayWaZFqCHkPVvJYITUPySf4vkhcBNvkWF2W+w4SujpqAE
 appendfile xAdZcMwfwCgUGpyAPeNfPxmQcEwVpDyhK+jfMWLB74swoq/1/k2EwKd4rq/PcOoC394=
 appendfile -----END CERTIFICATE-----
 
 move "__appendfile" "{parameter "storageDir" & "/TrustedDownloadCerts/testCert.crt"} 
 ]]></ActionScript>
 		</DefaultAction>
 	</Task>
 </BES>

Additional certificates

Starting from Patch 9, in addition to <StorageFolder>/TrustedDownloadCerts, a custom directory can be specified using the following parameter:

  • _BESRelay_Download_CaCertDirectory (server/relay). It is a string setting where you can declare the full path of a folder where the user stores its custom CA bundles. All the .crt and .pem files contained in this folder are added to the default certificates.
  • _BESClient_Download_CaCertDirectory (client). As described above, but the setting works for the client.

Disabling default certificates

Also, starting from Patch 9, a specific parameter can be set in order to ignore the default certificates and exclusively use the custom ones:

  • _BESRelay_Download_IgnoreDefaultCaCerts (server/relay). It is a boolean setting with two possible values: 0 and 1. When set to 1, it tells the component to ignore the default ca-bundle.crt certificate bundle located under BES Support. The default value is 0.
  • _BESClient_Download_IgnoreDefaultCaCerts (client). As described above, but the setting works for the client.

Summary

The following figure shows a summary of the locations used starting from Patch 9 to build the list of certificates. All contributions are appended to the certificate list, unless expressly ignored or empty.

For more details about download settings, see Download.

Troubleshooting hints

On Linux/UNIX platforms, the curl utility can be used to verify that a custom certificate bundle is valid and working for a specific download through HTTPS.

For example, on a Linux/UNIX server installation, the default certificate bundle can be tested with the following command, provided that the download URL is https://download.server.url/file-to-download:

curl --cacert 
/var/opt/BESServer/wwwrootbes/bfmirror/bfsites/enterprisemirror_<SiteID>_<SiteVersion>
/ca-bundle.crt -o file-to-download https://download.server.url/file-to-download

Note that you need to substitute the parameters <SiteID> and <SiteVersion> to get the actual directory name for the last version of the BES Support site.

To test a custom certificate bundle, the following command can be run (make sure to substitute the parameters with the appropriate values): 

curl --cacert </path-to-certificate/custom-certificate.pem> 
-o file-to-download https://download.server.url/file-to-download
Note: In some Windows versions, PowerShell maps the cmdlet Invoke-WebRequest to the curl executable; however, this does not provide a reliable method for certificate verification. Use openSSL instead, as described below.

On a Windows platform, the same test can be performed using openSSL. In the following example, download.server.url is the address of the HTTPS server and C:\path\to\certificate\certificate-bundle.pem is the local path to the certificate bundle.

openssl.exe s_client -connect download.server.url:443 -CAfile 
C:\path\to\certificate\certificate-bundle.pem
In this case, it is necessary to check the return code of the command in its output, which may be composed by many lines: the ones starting with “Verify return code” are those to look for. Possible examples are:
  • Verify return code: 0 (ok)
  • Verify return code: 20 (unable to get local issuer certificate)
  • Verify return code: 18 (self signed certificate)