NDES Installation and Configuration

The Network Device Enrollment Service (NDES) provides the SCEP interface that allows devices managed by BigFix MCM to request certificates from the Microsoft Certificate Authority. This section describes how to install and configure NDES and verify that it is ready to process certificate enrollment requests.

1. Install Network Device Enrollment Service (NDES)

NDES is installed as part of Active Directory Certificate Services (AD CS)

Steps

  1. Log in to the NDES server (domain-joined)
  2. Open Server Manager.
  3. Navigate to: Manage → Add Roles and Features
  4. Select Role-based or feature-based installation
  5. Select the target server.
  6. Under Server Roles, select Active Directory Certificate Services
  7. In Role Services, select Network Device Enrollment Service.
  8. Click Next and complete the installation.

2. Configure NDES Service Account

NDES requires a domain service account to request certificates from the CA.

Steps

  1. During the NDES configuration wizard, provide:
    • Service Account (example) corp\svc_ndes
  2. Ensure this account:
    • Is a domain user
    • Has permission to request certificates using the configured template (configured in CA section)

3. Configure CA Integration

NDES must be linked to the Certificate Authority and configured to use the correct certificate template.

Steps

  1. During configuration, select the target Certification Authority.
  2. Specify:
    • CA name (auto-discovered or manual selection)
  3. Ensure the following registry settings are configured (if required):

    Open:regedit

    Navigate to: HKEY_LOCAL_MACHINE\Software\Microsoft\Cryptography\MSCEP

    Key Configuration (Important)

    Set the certificate template name:
    EncryptionTemplate
GeneralPurposeTemplate
SignatureTemplate

    Example:

    EncryptionTemplate = BigFixSCEPTemplate
GeneralPurposeTemplate = BigFixSCEPTemplate
SignatureTemplate = BigFixSCEPTemplate

    Note:

    • The template name must match the template created in certtmpl.msc (not display name mismatch).
    • In most deployments, the same template is used for all three values.
    • NDES supports configuring different templates for each key type; however, this is not required for standard SCEP deployments and is not recommended unless there is a specific use case.

4. Configure HTTPS Binding for NDES

NDES endpoints should be secured using HTTPS to protect SCEP communication between BigFix, proxies, and the NDES server.
4.1 Obtain or Generate SSL Certificate

Ensure a valid SSL certificate is available for the NDES server.

Requirements:

  • Certificate must contain
    • the NDES server FQDN
    • the NDES Admin FQDN - to configure in pass-through mode for challenge password requests
  • Issued by:
    • Internal CA (recommended), or
    • Trusted public CA
  • Private key must be present on the server
Example:
4.2 Bind Certificate to IIS
  1. Open IIS Manager:
    inetmgr
  2. Navigate to: Sites → Default Web Site
  3. Click: Bindings (right panel)
  4. Click: Add
  5. Configure:
    • Type: https
    • Port: 443
    • Hostname
    • SSL Certificate: select the installed certificate
  6. Click OK

5. Verify NDES SCEP Endpoints

1. Verify Admin Endpoint

https://%3Cndes-server%3E/certsrv/mscep_admin

Expected:

  • Auth prompt
  • Challenge password displayed

This confirms:

  • Challenge password generation works
2. Verify Capabilities (GetCACaps)

https://%3Cndes-server%3E/certsrv/mscep/mscep.dll?operation=GetCACaps"

Expected response: A list of supported capabilities, such as:
POSTPKIOperation
SHA-1
SHA-256
AES
DES3

6. Configure and Verify NDES Challenge Password

NDES uses a one-time challenge password mechanism to authenticate SCEP enrollment requests.

NDES maintains a configurable pool of challenge passwords. This pool represents the number of available and unused OTPs at any given time. While it can be thought of as supporting concurrent requests, in practice it acts as a limit on how many unconsumed or in-progress enrollment requests can be handled simultaneously.

By default, NDES generates only a small number of challenge passwords, which may not be sufficient for enterprise environments. If the pool is exhausted, new enrollment requests may fail until existing OTPs are consumed or expire.

In real-world scenarios, OTPs may not always be consumed immediately. Delays, retries, or failed enrollments can result in unused or orphaned OTPs, which still occupy the pool until they expire. Therefore, it is important to size this setting appropriately.

Recommendation

Instead of using a fixed value, determine the appropriate size based on your environment:

  • Estimate the maximum number of devices that may request certificates concurrently
  • Account for retries, delays, and failed enrollments
  • Add buffer capacity to handle unused or orphaned OTPs

For example, if you expect around 75-100 concurrent enrollment requests, configure a higher value such as 150 to ensure sufficient capacity.

This approach helps prevent OTP exhaustion while maintaining reliable certificate enrollment at scale.

1. Increase Challenge Password Pool Size

By default, NDES generates only 5 challenge passwords, which may not be sufficient for enterprise environments.

Steps

  1. Open Registry Editor:
    regedit
  2. Navigate to:
    HKEY_LOCAL_MACHINE\Software\Microsoft\Cryptography\MSCEP
  3. Create a new key named:
    PasswordMax
  4. Under the PasswordMax key, create a new DWORD (32-bit) value with the name:
    PasswordMax
  5. Set value to:
    150

This increases the number of available challenge passwords.

2. Restart Services
After updating the registry, restart IIS:
iisreset
Note:
  • Increasing the password pool prevents request failures due to password exhaustion.
  • Challenge passwords are one-time use and time-bound.