Installing all required components on one computer

In this scenario, all components are installed on one computer. This type of deployment is best suited for demonstration or training deployments, not full production environments.

About this task

This scenario is divided into several sections:
Note:
  1. This scenario assumes that the SQL Server database is installed and configured so that key information is available during the configuration of the AppScan® Enterprise Server.
  2. If you are upgrading from a previous version of AppScan Enterprise, read Replacing Jazz Team Server with WebSphere Liberty - Frequently asked questions before you begin upgrading.
  3. To migrate Jazz Team Users users to this new authentication method, export a .csv file of users by using the cd <install-dir>\Appscan Enterpise\JazzTeamServer\server\ repotools-jts.bat -exportUsers toFile=C:\users.csv repositoryURL=https://<hostname>:9443/jts before you begin upgrading to v9.0.1. Then follow the steps in this topic: Configuring a basic user registry for the Liberty profile to import the users into Liberty.

Licensing information for AppScan® Enterprise

AppScan Enterprise 10.8.0 no longer supports FlexNet licensing (FNO). Instead, licensing must be configured through the My HCLSoftware (MHS) portal. With this change, AppScan Enterprise now manages licenses within the application, eliminating reliance on the Local License Server (LLS) and Cloud License Server (CLS). If upgrading from AppScan Enterprise 10.6.0 or earlier, decommission the Local License Server (LLS). Refer to the KB article KB0119306 for instructions on uninstalling LLS. To learn more about the MHS portal, refer to MHS help.

Configuring licensing in HCL® AppScan® Enterprise via MHS

This task covers the steps to configure licensing for HCL AppScan Enterprise via My HCLSoftware (MHS) portal, including creating a deployment and allocating features.

Before you begin

Make sure you have access to the MHS portal.

Procedure

  1. Create a deployment in MHS:
    1. Navigate to MHS portal > Deployment.
    2. Click Create Deployment.
    3. Enter a deployment name and select AppScan Enterprise from the product list.
    4. Click Add Deployment.
  2. Allocate features to the deployment:
    1. Open the newly created deployment.
    2. Review and allocate the available AppScan Enterprise features, including:
      • AppScan Enterprise Dynamic Analysis Authorized User
      • AppScan Enterprise Dynamic Analysis Floating User
      • AppScan Enterprise Dynamic Analysis Scanner Install
      • AppScan Enterprise Interactive Analysis Install
      • AppScan Enterprise Reporting Only Authorized User
      • AppScan Enterprise Reporting Only Floating User
      • AppScan Enterprise Server Basic Install
    3. Enter the required allocation count for each feature.
    4. Once done, click Save all changes.
      Note:
      • The count for AppScan Enterprise Interactive Analysis Install (IAST) must be entered in multiples of 4.
      • Either AppScan Enterprise Server Basic or AppScan Enterprise Server Install must have an allocation for the license generation to proceed.
      • If these conditions are not met, an error will occur during license download.

Installing HCL® AppScan® Enterprise Server

Use this procedure to install the User Administration component and Enterprise Console for reporting and user administration tasks.

Before you begin

Make sure you read the Required user account information during installation and configuration so that you know which user account to use during installation.

Procedure

  1. Go to the directory where you downloaded the compressed file (AppScanEnterpriseServerSetup_<version>.zip), extract the files, and double-click the AppScanEnterpriseServerSetup_<version>.exe file.
    Note:
    It might take a while for the next screen to display. The compressed file includes these files:
    • AppScanEnterpriseServerSetup_<version>.exe
    • HCL AppScan Enterprise Server.msi - do not run this file
    • Data1.cab
  2. In the Setup wizard Welcome screen, click Next.
  3. In the License Agreement window, select the I accept the terms in the license agreement option, and click Next.
  4. In the Destination Folder window, do one of the following actions and click Next:
    1. Click Next to accept the default installation location.
    2. Click Change to select a different installation location.
  5. In the Ready to Install the Program window, click Install to proceed with the installation.
  6. On the Setup Wizard Completed screen, click Finish to launch the Configuration Wizard.

Running the Configuration wizard to get the fingerprint key

After you install or upgrade the Server or Scanner, you must configure each installed component and run the Configuration wizard on all instances and all servers. AppScan Enterprise uses MHS for licensing. The licensing process requires a Fingerprint Key, which is generated when running the Configuration Wizard. If you need to generate this key before you run the wizard, you can use a standalone utility described in Generating a machine fingerprint for AppScan Enterprise.

Before you begin

  1. During configuration, you define the name and location of the SQL Server database to be used, and the service account name and password. The user who runs the configuration wizard must be able to create a database and grant rights.
  2. If you encounter an error "**WARNING** Unable to configure virtual directory "ase" for local directory "C:\Program Files (x86)\HCL\AppScan Enterprise\WebApp". Ensure IIS is configured properly and try again, consider disabling your antivirus software while you are running the configuration wizard. If you do not want to disable the antivirus software, you can exclude the AppScan Enterprise folder from the antivirus configuration, and run the configuration wizard again.
    Important:
    Personal firewalls and Antivirus programs running on the same computer where AppScan Enterprise is running may block communication and result in configuration failure during installation and configuration. It also results in inaccurate findings and reduced performance while running the scans. For best results do not run firewalls or antivirus on the computer.

Procedure

  1. When the installation is complete, the Configuration wizard launches automatically. You can also start it by selecting Configuration Wizard from the Windows™ Start menu as an administrator.
  2. In the Welcome screen, click Next.
  3. In the License Server window, copy the Fingerprint Key displayed on the screen. This key is unique to your machine and is required to generate a valid license.

Generating and applying the license file

This task covers the steps to generate and apply a license file for AppScan Enterprise, including generating, configuring, and placing the license file.

Before you begin

Ensure that you have a deployment with an allocated license in MHS. Refer to Configuring licensing in HCL AppScan Enterprise via MHS

Procedure

  1. Generating the license file:
    1. Go to MHS portal > Deployment.
    2. Find your existing deployment and click the three-dot menu next to the deployment.
    3. Click Download License File.
    4. In the Download License File window, paste the Fingerprint key (copied from the Configuration wizard or the standalone utility).
    5. Once done, click Submit to download the license file.
  2. Applying the license file:
    1. Copy the downloaded license file to the AppScan installation directory (e.g., C:\Program Files (x86)\HCL\AppScan Enterprise).
    2. The file name must be AppScanEnterprise_License.license.
    Note:
    Whenever the AppScanEnterprise_License file is modified, such as a license expiry, or any other change, AppScan Enterprise does not automatically recognize the update. To ensure proper functionality, restart the AppScan Enterprise services after modifying the license file.

Completing the Configuration wizard

After applying the license, follow the steps to complete the configuration of AppScan Enterprise. You must configure each installed component and run the Configuration wizard on all instances and all servers.

Before you begin

Ensure that the license file has been placed in the AppScan installation directory. Refer to Generating and applying the license file.

Procedure

  1. Open the configuration wizard again as an administrator from the Windows™ Start menu or from the installation directory (e.g., C:\Program Files (x86)\HCL\AppScan Enterprise\WFCfgWiz.exe)
  2. In the Welcome screen, click Next.
  3. In the License Server window, click Next.
  4. In the Server Components window, select the components that you want to configure. The components available to you depend on your license. See Server Components. If you are installing the components on one machine, select all the checkboxes, even if you have installed one of the components previously.
  5. In the Instance Name window, specify the name of the instance you want to configure. See Instance Name.
  6. In the Service Account window, enter the Domain/Username Service Account and password, and click Next. See Service Account.
  7. In the Database Connection window, enter the SQL Server name, port number, select the authentication mode and the name of the database you are connecting to. If you select SQL Authentication, enter the username and password before you click Test Connection to verify that you can connect to the SQL Server. The configuration wizard does not proceed until the connection is successful. When AppScan® Enterprise Server creates the database in SQL Server, it automatically configures the collation for it.
    Note:
    1. AppScan Enterprise supports Windows Authentication and SQL authentication to connect to SQL Database. Refer to Microsoft's article on Authentication Mode and choose the authentication mode that fits your requirements. You can change from one authentication mode to another, but make sure to re-run the configuration wizard.
    2. The user must be assigned a sysadmin server role on the SQL server. The user must also have access to the database and must be granted a minimum of db_ddladmin, db_datawriter, and db_datareader rights or the db_owner rights.
    3. The syntax for the SQL Server name has changed with the introduction of Liberty support. ".\SQL_SERVER_NAME" no longer works. Use "HOSTNAME\SQL_SERVER_NAME" instead.
    4. If you are upgrading an existing database from v8.6 or earlier, enter the Database Master Key Password on the next screen to access it. Keep this password in a secure location.
    5. If your environment uses a named SQL Server instance for the AppScan Enterprise database, make sure that TCP/IP is enabled in the SQL Server configuration manager, and restart the SQL services for the SQL Server. Use the port number of the named SQL Server instance instead of the default port number (1433).
    6. If you use a raw IPv6 address for the SQL Server, you must enclose the address in brackets (for example, [2001:db8::1]).
  8. In the Server Certificate window, choose a certificate specific to your organization. This step helps you deploy a secure AppScan® Enterprise in your environment. See Server Certificate.
  9. (Upgrade only). In the Restore AppScan Server Settings screen, you can choose to restore the previous AppScan Server customized settings on the Liberty Server (default). This screen appears once upon upgrade; if you run the configuration wizard later, this screen won't appear. See Restore AppScan Server settings.
  10. In the Server Keystore screen, select a server keystore to be used by the Enterprise Console. If you exported a .pfx file, select Public key cryptography standards #12 (PKCS #12). Browse to the location where you saved the .pfx file, import it and enter the password you created when you exported the file. See Server Keystore.

  11. In the Authentication Mechanism window, select an Authentication Mechanism to use to log in to the Enterprise Console. The default is to authenticate via Windows™. To use LDAP, see Authentication Mechanism.
    Note:
    If you need to authenticate with the Common Access Card (CAC), make sure you choose LDAP as your authentication mechanism. Once AppScan Enterprise is configured, follow the instructions in Authenticating with the Common Access Card (CAC) to authenticate with CAC.
  12. In the Server Configuration window,

    1. Configure the hostname and port of the Liberty server for AppScan Server to use. If you are using Windows authentication, prefix the hostname with your domain name.
    2. While it is not a recommended practice, you can allow SSL connections with invalid or untrusted certificates during scanning. When the option is disabled, messages will appear in the scan log to indicate that the insecure server cannot be reached for scanning. This option also affects the Manual Explore functionality.
    3. : Configure the How to Fix services port (installed with the Enterprise Console). This port runs over HTTP and is used by the ASRA server to provide How to Fix. The How to Fix appears on the About this Issue page for the application issues and provides fix recommendations. You can select a different port if 9444 is already used.
  13. (Upgrade only) In the Database Encryption Changes window, click Help to learn how to protect the SQL Server where the database is located. If you decide not to enable TDE, select the check box so you can continue configuration.

    Note:
    AppScan® Enterprise uses transparent data encryption (TDE) technology that is available in SQL Server 2016 and later. TDE encrypts the data that is stored in the database or in backups on physical media. If you are using an older version of SQL Server, any data that is contained in that database is at risk of compromise by unauthorized access.
  14. In the Product Administrator window, specify a user as a Product Administrator. This user is licensed separately; if you want to reassign the Product Administrator license, you must rerun the configuration wizard. See Product Administrator.

  15. Ensure that nobody is accessing the database, and click Finish in the Specifications Complete window to complete the configuration. This process might take a while.
    Note:
    1. IIS AppPool settings on Windows™ Server are set during configuration:
      • IIS recycling is set at 2:00am
      • Idle timeout is set at 120 minutes
    2. If you see an error message that the proxy server certificate cannot be configured, it might be expired. Contact your Product Administrator to investigate further.
  16. Optional: Select the Start the Services check box to automatically start the services.
    Note:
    If you do not choose to automatically start the agent service, the agents do not pick up any jobs that are created by users. You can manually start the service by using the Administrative tools; see Verifying the agent service and alerting service installation.
  17. Run the Default Settings Wizard. This wizard helps you to install sample data in by providing defaults for a number of configurable options.
  18. Click Exit.

Running the Default Settings wizard

This wizard helps you install sample data in by providing defaults for a number of configurable options. You can create users, add security test policies, create scan templates, add pre-created dashboards, and configure defect tracking integration with Rational® Quality Manager or Rational® Team Concert.

About this task

Ensure that the Launch Default Settings Wizard check box is selected when the Configuration wizard finishes.

Procedure

  1. In the Welcome page, choose the instance that you want to update, and click Next.
  2. In the Initialization Type window, select one of the available initializations, and click Next.
  3. In the Default Setting window, configure the following options and click Next:
    1. Instance: Select the instance name for this setup. The Instance that was configured in the Configuration wizard is selected here by default.
    2. Contact: The name or a point of contact for the items that are created by the wizard. You can edit these items later if necessary.
    3. Root folder name: Enter a name for the default root folder. The default folder acts as the root folder for all other folders you create.
    4. Application URL: Enter the URL for the application users to access the application. By default, this URL is the current computer's FQDN (fully qualified domain name).
      (for example, http://myserver/mydomain/appscan/).
  4. (Windows™ authentication only): In the LDAP Settings page, select the Enable LDAP check box if you use an LDAP server.
    1. In the Server Name field, enter the LDAP group name.
    2. In the Group Query field, enter the path of the group query that is used to retrieve user group information. You can use an LDAP server or an Active Directory server.
    3. Optional: If you want to integrate with the LDAP server by using anonymous access, select the Anonymous access check box. This option is disabled by default.
    4. Click Test LDAP to confirm the configuration works.
  5. In the IP Security Permissions page, configure the IP addresses and ranges that are allowed for scanning. Use a dash to define IPv4 ranges (such as 1.2.3.4 - ); use a prefix to define IPv6 ranges (such as fe80::/10).
  6. In the Populate Database with Sample Data page, select the Populate Sample Data check box to populate the database with scan templates, pre-created dashboards, server groups, and test policies.
  7. Click Next. The Default Settings Wizard Progress page opens, displaying the setup's progress.
  8. When the wizard is complete, the Default Settings Wizard Complete page opens.
  9. Click Exit to close the wizard.

Installing HCL® Dynamic Analysis Scanner

Use this procedure to install the agents that are used for scanning and testing your website applications.

Before you begin

Note:
  1. Make sure you read the Required user account information during installation and configuration so that you know which user account to use during installation.
  2. Any technologies that you use on your website must also be installed with the Scanner. For example, if you use Flash on any web pages, you must have the correct version of Flash installed.

Procedure

  1. Go to the directory where you downloaded the executable file (ASE_DASSetup_<version>.exe) and double-click the file.
    Note:
    It might take a while for the next screen to display.
  2. In the License Agreement window, select the I accept the terms in the license agreement option, and click Next.
  3. In the Destination Folder window, click Next.
  4. In the Ready to Install the Program window, click Install to proceed with the installation, and then click Finish.

Running the Configuration wizard

After you install or upgrade the Server or Scanner, you must configure each installed component and run the Configuration wizard on all instances and all servers.

Before you begin

  1. During configuration, you define the name and location of the SQL Server database to be used, and the service account name and password. The user who runs the configuration wizard must be able to create a database and grant rights.
  2. Running the wizard after you install the AppScan® Enterprise Server sets up the database on the SQL Server and does the initial setup of the component.
  3. Running the wizard after you install the Dynamic Analysis Scanner registers the Scanner with AppScan® Enterprise Server.
  4. Ensure that the license file (AppScanEnterprise_License) has been placed in the AppScan installation directory (e.g., C:\Program Files (x86)\HCL\AppScan Enterprise). Refer to Generating and applying the license file.

Procedure

  1. When the installation is complete, the Configuration wizard launches automatically. You can also start it by selecting Configuration Wizard from the Windows™ Start menu.
  2. In the Welcome screen, click Next.
  3. In the License Server window, click Next.
  4. In the Server Components window, select the components that you want to configure. The components available to you depend on your license. See Server Components. If you are installing the components on one machine, select all the checkboxes, even if you have installed one of the components previously.
  5. In the Instance Name window, specify the name of the instance you want to configure. See Instance Name.
  6. In the Service Account window, enter the Domain/Username Service Account and password, and click Next. See Service Account.
  7. In the Database Connection window, enter the SQL Server name, port number, select the authentication mode and the name of the database you are connecting to. If you select SQL Authentication, enter the username and password before you click Test Connection to verify that you can connect to the SQL Server. The configuration wizard does not proceed until the connection is successful. When AppScan® Enterprise Server creates the database in SQL Server, it automatically configures the collation for it.
    Note:
    1. AppScan Enterprise supports Windows Authentication and SQL authentication to connect to SQL Database. Refer to Microsoft's article on Authentication Mode and choose the authentication mode that fits your requirements. You can change from one authentication mode to another, but make sure to re-run the configuration wizard.
    2. The user must be assigned a sysadmin server role on the SQL server. The user must also have access to the database and must be granted a minimum of db_ddladmin, db_datawriter, and db_datareader rights or the db_owner rights.
    3. If you are upgrading an existing database from v8.6 or earlier, enter the Database Master Key Password on the next screen to access it. Keep this password in a secure location.
    4. If your environment uses a named SQL Server instance for the AppScan Enterprise database, make sure that TCP/IP is enabled in the SQL Server configuration manager, and restart the SQL services for the SQL Server. Use the port number of the named SQL Server instance instead of the default port number (1433).
  8. (upgrade only) In the Database Encryption Changes window, click Help to learn how to protect the SQL Server where the database is located. If you decide not to enable TDE, select the check box so you can continue configuration.

    Note:
    AppScan® Enterprise uses transparent data encryption (TDE) technology that is available in SQL Server 2016 and later. TDE encrypts the data that is stored in the database or in backups on physical media. If you are using an older version of SQL Server, any data that is contained in that database is at risk of compromise by unauthorized access.
  9. Ensure that nobody is accessing the database, and click Finish in the Specifications Complete window to complete the configuration. This process might take a while.
    Note:
    1. IIS AppPool settings on Windows™ Server are set during configuration:
      • IIS recycling is set at 2:00am
      • Idle timeout is set at 120 minutes
    2. If you see an error message that the proxy server certificate cannot be configured, it might be expired. Contact your Product Administrator to investigate further.
  10. Optional: Select the Start the Services check box to automatically start the services.
    Note:
    If you do not choose to automatically start the agent service, the agents do not pick up any jobs that are created by users. You can manually start the service by using the Administrative tools; see Verifying the agent service and alerting service installation.
  11. Click Exit.

Verifying the installation of the Enterprise Console

After the installation process is complete, you can verify the installation of the Enterprise Console.

Procedure

Go to https://domain/ase/ and log in.