Enabling FIPS 140-2 or NIST SP800-131a on WebSphere Liberty Profile

Use one of these procedures to enable FIPS 140-2 or NIST SP800-131a on WebSphere Liberty Profile.

Before you begin

Run the configuration wizard and start the services before you start this task.

Procedure

  1. To enable FIPS 140-2:
    1. Locate the installation directory of Liberty at <install-dir>\AppScan Enterprise\Liberty\usr\servers\ase.
    2. Add the -Dcom.hcl.jsse2.usefipsprovider=true property to the jvm.options file to enable the JSSE2 provider to run in FIPS 140-2 mode.
    3. Go to <install-dir>\AppScan Enterprise\Liberty\jre\lib\security directory.
    4. In a text editor, edit the java.security master security properties file to register additional cryptographic package providers.
    5. Update these two lines:

      #ssl.SocketFactory.provider=
      #ssl.ServerSocketFactory.provider=

      to

      ssl.SocketFactory.provider=com.hcl.jsse2.SSLSocketFactoryImpl
      ssl.ServerSocketFactory.provider=com.hcl.jsse2.SSLServerSocketFactoryImpl

    6. Locate the list of cryptographic providers that are located after the line # List of providers and their preference orders and replace it with the following list:

      security.provider.1=com.hcl.crypto.fips.provider.HCLJCEFIPS
      security.provider.2=com.hcl.jsse2.HCLJSSEProvider2
      security.provider.3=com.hcl.crypto.provider.HCLJCE
      security.provider.4=com.hcl.security.jgss.HCLJGSSProvider
      security.provider.5=com.hcl.security.cert.HCLCertPath
      security.provider.6=com.hcl.security.sasl.HCLSASL
      security.provider.7=com.hcl.xml.crypto.HCLXMLCryptoProvider
      security.provider.8=com.hcl.xml.enc.HCLXMLEncProvider
      security.provider.9=org.apache.harmony.security.provider.PolicyProvider
      security.provider.10=com.hcl.security.jgss.mech.spnego.HCLSPNEGO

    7. Go to <install-dir>\AppScan Enterprise\Liberty\jre\bin and open a cmd window. Your certificates must be at least 1024 in size and can be signed with a DSA or RSA signature algorithm. The keytool utility can be used to generate a compatible keypair: 1 keytool -genkey -alias default -keyalg RSA -keysize 1024 -dname CN=example -keystore fips.jks -storepass Liberty -keypass Liberty.
    8. Save and close the file, and then rerun the configuration wizard.
  2. To enable NIST SP800-131a:
    1. Locate the installation directory of Liberty at <install-dir>\AppScan Enterprise\Liberty\usr\servers\ase.
    2. Add the -Dcom.hcl.jsse2.sp800-131=transition property to the jvm.options file to enable the JSSE2 provider to run in NIST transition mode.
    3. Go to the server.xml file in the same directory and replace the sslProtocol="SSL_TLSv2" property with sslProtocol="TLSv1.2".
    4. Save and close the file, and then rerun the configuration wizard.