RHSM troubleshooting checklist
Troubleshooting RHSM errors
What to check | Errors or warnings encountered | Possible causes and remediation steps |
---|---|---|
GPG key is imported. | If the GPG is not imported in an endpoint, you might find this entry in the EDR log:
warning: rpmts_HdrFromFdno: Header V3 RSA/SHA256 Signature, key ID fd431d51: NOKEY Public
key for httpd-devel-2.2.15-56.el6_8.3.x86_64.rpm is not installed . |
The GPG is not imported in an endpoint. Check the EDR log of the endpoint. The log displays if the GPG is not imported. If it is not, import the GPG through the command line or by using Fixlets specific to either the Patches for RHEL 6 Native tools site or the Patches for RHEL site. |
The prefetch plug-ins timeout settings is set too low. | You might encounter a failed Fixlet deployment and with "fail" indicated at the
line. |
Use a task to set the timeout to 30 minutes. |
Ensure that your certificates can access Red Hat repositories. |
Error: Certs cannot access any Repos or the certificates are only able to access
some of the required repositories. |
Run a quick repository access check. |
Ensure that the entitlement certificates are placed in the correct folders. | Patch deployment fails. | The certificates might not be placed in the correct folders and sub-folders. Unnecessary metadata files must be removed. For more information, see the guidelines in the following section: Entitlement certificates and system identity certificate are placed in the correct folders. |
Ensure that the entitlement certificates have the correct format. | Patch deployment fails. | The user might have entitlements that have the old formats. To check the certificate format, see the steps detailed in the following section: The version of RHSM entitlement certificates have the correct format. |
Entitlement certificates are active and have not expired. | Patch deployment fails one day after creating an entitlement certificate. | Follow the methods in the section to verify that the certificates have not expired. If the subscription is expired, you must generate or attach a new subscription (entitlement) to the entitlement certificate. Regenerate the identity certificate if it is expired. |
Entitlement certificates have the correct subscriptions (entitlements) attached. | Certificates cannot access the required Red Hat base repositories. | Attach the correct subscription (entitlement) to the Entitlement Certificate to get access to the required repositories. Follow the methods in the section to verify the subscription entitlements that are attached to your entitlement certificates. |
Entitlement certificate can access the Red Hat base repositories. | Certificates cannot access the Red Hat base repositories. | Possible causes:
|
Error messages in RHSMPlugin.log | You might find the following entry in the log:
|
|
The GPG key is imported
A GPG key must be imported from Red Hat to download Red Hat content. After deploying a patch,
check the EDR log of the endpoint, which is located at
var\opt\BESClient\EDRDeployData\EDR_DeploymentResults.txt
.
If the GPG key for an endpoint is not imported, you might see the following warning in EDR_DeploymentResults.txt.
warning: rpmts_HdrFromFdno: Header V3 RSA/SHA256 Signature, key ID fd431d51:
NOKEY Public key for httpd-devel-2.2.15-56.el6_8.3.x86_64.rpm is not installed
To import the GPG key for the endpoint, use the following command: rpm --import
/mnt/cdrom/RPM-GPG-KEY-redhat-release
.
- Patches for RHEL 6 Native Tools: 301 Import RPM-GPG-KEY-redhat-release - RHEL 6
- Patches for RHEL 7: 301 Import RPM-GPG-KEY-redhat-release - RHEL 7
Ensure that the timeout setting is sufficient to execute the prefetch plug-in
You might need to configure the plug-ins timeout setting if the Fixlet deployment fails and from
the Action Script Execution Detail in the console, "fail" is indicated in the execute
prefetch plug-in
line.
From the Patching Support site, use this task to set the timeout to 30 minutes: Change Timeout for Prefetch Plugins.
After applying the task, restart the BES client with the following task from the BES Support site: TROUBLESHOOTING: Restart BES Client on RHEL/SUSE.
Your certificates can access Red Hat repositories
You can run a quick test to check if your certificates can access Red Hat repositories. Typically, the test runs less than 10 seconds.
The RHSM plug-in is usually located in C:\Program Files (x86)\BigFix Enterprise\BES Server\DownloadPlugins\RHSMProtocol.
RHSMPlugin.exe --check-baserepos
There are three possible outcomes when running the repository access test. Possible outcomes from Repo Access Test:
- None of the certs can be accessed
-
The certificates were not set up properly. Continue with the checklist. For more information about setting up and downloading both certificates, see Setting Up RHSM Certificates.INFO : Base Repos Test Summary INFO : Certs in <rootCertDir> can access 0 / 12 Base Repos: ERROR : Error: Certs cannot access any Repos.
- Certificates are able to access all required repositories.
-
INFO : Base Repos Test Summary INFO : Certs in <rootCertDir> can access 3 / 12 Base Repos: INFO : server-7-x86_64: Red Hat Enterprise Linux 7 Server (RPMs) INFO : server-6-x86: Red Hat Enterprise Linux 6 Server (RPMs) INFO : server-6-x86_64: Red Hat Enterprise Linux 6 Server (RPMs)
- Certificates are able to access only some required repositories.
You might need to patch endpoint types that do not appear in the list of successfully accessed repositories. For example, you have Workstaton endpoints but the output only shows access to the Server repositories. In such cases, you must attach the required subscriptions to the certificates through the Red Hat portal.
The list of repositories that are tested are derived from the RepoList (“primaryRepoListFile", “extendedRepoListFile") file that is referenced in the plugin.ini. At the time of writing, the list of repositories in1
are as follows.- client-6-x86
- client-6-x86_64
- client-7-x86_64
- server-6-x86
- server-6-x86_64
- server-7-x86_64
- workstation-6-x86
- workstation-6-x86_64
- workstation-7-x86_64
- server-6-s390x
- server-7-s390x
- server-7-ppc64le
- server-7-ppc64be
Entitlement certificates are placed in the correct folders
Ensure that the certificates are in the correct folders. Follow these guidelines to avoid errors.
- The following path is the relative path where the rootCertDir is located. This can be left at
its default value ("certs") in the
plugin.ini
^^^^^^^^^^^^^^^^^^^^^^^^ rootCertDir = certs ^^^^^^^^^^^^^^^^^^^^^^^^
- The "certs" folder must only contain subfolders. For example, cert_set_1, cert_set_. Remove metadata files.
- Within the "cert_set_1" folder only files ending with ".pem" are allowed. There can be any no.
of Entitlement Certificates in "cert_set_1". For example, 443229635427054308.pem. Only Entitlement
Certificates with the new format are allowed.Note: Earlier versions of the RHSM subscription interface had an option to download the system identity certificate. This is no longer the case with the current RHSM subscription interface version. The System Identity Certificate is no longer required from v1.0.2.0 of the RHSM download plug-in and RHSM download cacher.
- If you have more than one set of certificates, ensure that only one set of certificates go to one folder.
The version of RHSM entitlement certificates have the correct format
When creating RHSM certificates in the RHSM customer portal, and you are at the step in which you need to register a system, you must specify the Red Hat Enterprise Linux version.
To avoid errors, select version 7.2. Red Hat Enterprise Linux versions that are earlier than version 7.2 have a different entitlement certificate format that the RHSM download plug-in does not read. The new entitlement certificate format has "BEGIN ENTITLEMENT DATA" in the .pem file.
- Open the .pem file in a text editor.
- Search for "BEGIN ENTITLEMENT DATA". Only the new entitlement format will have this; neither the old entitlement certificate format and the system identify certificate will have "BEGIN ENTITLEMENT DATA".
The entitlement certificates are active and have not expired
- Through a Red Hat machine.
- Through the Red Hat portal. You must access the account in https://access.redhat.com that generated the entitlement certificates.
- Through OpenSSL
- Verifying the correct subscription (certificate) attachments using a Red Hat machine
-
From the command line in a Red Hat machine, run
> rct cat-cert <entitlement cert> > output.txt
to print the certs metadata to the ouput.txt file. Repeat this for each Entitlement Certificate and the System Identity Certificate using a different output.txt filename. Open the file in a text editor and the certs expiry date will be in the End Date fieldEnd Date: 2018-05-25 12:50:11+00:00
.
- Verifying the correct subscription (certificate) attachments through the Red Hat portal
-
- Log in to https://access.redhat.com.
- Go to https://access.redhat.com/management/consumers?type=system
- Click the system that you previously created. A list of Entitlement Certificates displays.
- For each Entitlement Certificate, click View. Go to the Order Info tab. In the End Date column, verify that the subscription are not expired.
- Click Back in the browser and repeat steps 4 to 6 for each Attached Subscription in your system.
- Verifying the correct subscription (certificate) attachments through OpenSSL
-
If you are able to use openssl, open a command line at this folder:
Use this command and replace the name of the “.pem" file:
$ openssl x509 -enddate -noout -in 7a8337a5-eb47-4a52-a161-9635d5691996.pem
This results to the expiry date of the certificate. For example,notAfter=Jan 10 15:19:14 2018 GMT
Entitlement certificates with the attached subscription (entitlement) with Name: Red Hat
Enterprise Linux for Virtual Datacenters
has been known to stop working after 1 day. If you
are having issues with patch deployment after 1 day, we suggest avoiding this subscription and using
a non-Virtual Datacenters subscription like Red Hat Enterprise Linux 7 Server
(RPMs)
instead.
Entitlement Certificates have the correct Subscriptions (Entitlements) attached
- Verifying the correct subscription (certificate) attachments using a Red Hat machine
-
From the command line in a Red Hat machine, run
> rct cat-cert <entitlement cert>
. This displays the entitlement certificate metadata, including the expiry date of the certificate and the repositories that the certificate can access.In the following example, Certificate:
End Date: 2017-01-17 13:30:47+00:00
shows the expiry date. Content:Name: Red Hat Enterprise Linux 7 Server (RPMs)
lists the repositories that the certificate it can access.==========================================
Certificate:
Path: 7a85f98153c2eb950153c73d2fb159e5.pem
Version: 3.2
Serial: 3689711437028903897
Start Date: 2016-03-31 04:00:00+00:00
End Date: 2017-01-17 13:30:47+00:00
Content:
Type: yum
Name: Red Hat Enterprise Linux 7 Server (RPMs)
Label: rhel-7-server-rpmsVendor: Red Hat
URL: /content/dist/rhel/server/7/$releasever/$basearch/os
GPG: file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release
Enabled: True
Expires: 86400
Required Tags: rhel-7-server
Arches: x86_64
==========================================
You might need the following base repositories, depending on the endpoint that you deploy patches to.- Red Hat Enterprise Linux 6 Desktop (RPMs)
- Red Hat Enterprise Linux 6 Workstation (RPMs)
- Red Hat Enterprise Linux 6 Server (RPMs)
- Red Hat Enterprise Linux 7 Desktop (RPMs)
- Red Hat Enterprise Linux 7 Server (RPMs)
- Red Hat Enterprise Linux 7 Workstation (RPMs)
Name: Red Hat Enterprise Linux for Virtual Datacenters
has been known to stop working after 1 day. If you are having issues with patch deployment after 1 day, we suggest avoiding this subscription and using a non-Virtual Datacenters subscription likeRed Hat Enterprise Linux 7 Server (RPMs)
instead. - Verifying the correct subscription (certificate) attachments by accessing the account on https://access.redhat.com that generated the entitlement certificates
- Log in to https://access.redhat.com.
- Go to https://access.redhat.com/management/consumers?type=system
- Click the system you previously created. A list of list of attached subscriptions displays.
- For each subscription, in the Entitlement Certificate column, click .
- Click Back in the browser and repeat Step 4 for each Attached Subscription in your system.
- Open each export.CSV that was downloaded from RedHat. Under the Name column, search for the Base repository name of the repositories that you need access to for patch deployment.
- Red Hat Enterprise Linux 6 Desktop (RPMs)
- Red Hat Enterprise Linux 6 Workstation (RPMs)
- Red Hat Enterprise Linux 6 Server (RPMs)
- Red Hat Enterprise Linux 7 Desktop (RPMs)
- Red Hat Enterprise Linux 7 Server (RPMs)
- Red Hat Enterprise Linux 7 Workstation (RPMs)
Entitlement certificate can access the Red Hat base repositories
- Test the access to the Red Hat base repositories.
- Determine if the proper subscriptions have been attached to the entitlement certificate.
- For RHSMPlugin.exe (v1.0.0.2 and later): >>>RHSMPlugin.exe --check-baserepos
- For RHSMDownloadCacher.exe (v1.0.0.2 and later) : >>> RHSMDownloadCacher.exe --rootCertDir certs check-baserepos
- The certificates have expired. To remedy this, see CHECK 5: Entitlement Certificates and System Identity Certificate are not expired.
- The required subscription were not properly attached when the system was registered through Red Hat Subscription Management portal. To remedy this, see CHECK 4: Entitlement Certificates have the correct Subscriptions (Entitlements) attached.
- The network or proxy is blocking RHSMPlugin.exe from accessing the repositories. Check that your network firewall or proxy is not blocking the RHSMPlugin.exe. If the problem persists, you might need to contact Support.
Error messages in RHSMPlugin.log
The RHSMPlugin.log is located in <BES_Server>\DownloadPlugins\RHSMProtocol\logs.
You might encounter the following error in the log:
ERROR : All Key and Cert pairs in 'rootCertDir' cannot access:
https://cdn.redhat.com/content/dist/rhel/client/7/7Client/x86_64/os/repodata/repomd.xml
This error message indicates that RHSMplugin was not able to access RedHat’s Client RHELClient 7 Repo (“rhel/client/7/7Client/x86_64/os").
This happens when the same package is found in multiple repositories. This will prompt the RHSM download plug-in to access all the repositories where the package is located. When the download plug-in tries to access a repository that it does not have access to and when there is a lack of entitlement of the Entitlement Certificate, it will the indicate the error in the log.
- If you do not need to deploy patches to any endpoints, as for example, RHEL Client 7 machines
-
If you do need to deploy patches to any such endpoints, you may safely ignore this message. This error might be due to the client script nohash limitation which the RHSM download plug-in cannot avoid. For more information, see: https://developer.bigfix.com/action-script/reference/download/add-nohash-prefetch-item.html.
- If the error message is associated with a repository that you need for your patch deployment, for example if you have a RHEL 7 Server endpoint that requires patching
- This might be caused by any of the following reasons:
- The certificates have expired or were revoked. To remediate this, see the steps in "Ensure that entitlement certificates are not expired."
- The required subscriptions were not properly attached when registering the system through RedHat. To remediate this, see the steps in "Entitlement certificates are not able to access the repositories."
- The network or proxy is blocking the RHSMPlugin.exe from accessing the repositories. Check that your network firewall or proxy is not blocking the RHSMPlugin.exe. If the problem persists, you might need to contact Support.