Evaluate Explore results

Before proceeding to the Test stage, review the Explore results, because if important areas of the site were missed during the Explore stage, they will not be tested during the Test stage.

About this task

The results of the Explore stage are shown in the three Data view panes. The following are some tips for evaluating whether the Explore stage worked well and achieved good coverage of the application.
Note: If you make any configuration changes during this stage, you should re-Explore the application before starting the Test stage.

Procedure

  1. Scan Log. Use this to check that AppScan wasn't out-of-session too often.
    1. Click View > Scan Log,
    2. Scroll down the log entries to check that AppScan wasn't out-of-session too often.

    If AppScan was out-of-session more than a couple of times in five minutes, it may help to re-record and reconfigure the recorded login, paying special attention to the in-session detection configuration.

  2. Application Tree. This is a graphical representation of all the areas of the site that were discovered and explored. Use it to see how well the site was covered.
    1. Does the application tree accurately show the hierarchical structure and main pages of your application?
    2. Are there Login URLs in the tree? (If there are not, the login was never sent.)
    3. Does the total number of visited URLs (bottom left-hand corner) match your understanding of the actual site size?
    4. Were a reasonable number of tests created, for sending during the Test stage (should be at least five times the number of URLs)?
  3. Requests sent. Review and validate the requests sent during the Explore stage.
    1. In the Data pane, select Requests view, to display all requests sent.
    2. Verify that the login URLs appear in this list, particularly the in-session request and the login request containing the user credentials.
    3. Look over some of the requests that appear after the login request in the Login procedure. Verify that the response does not contain errors. To do this, type the word "error" into the Detail pane search field, and then select URLs one by one in the upper panel. If a particular response contains the word "error", the search field color will change from red ("not found"), to green ("found"), and the word "error" will be highlighted in the response body.
    4. If these requests contain error strings, this indicates that the user was out-of-session, and therefore that the Login procedure was not correctly recorded. Record it again.
  4. Application Data view. This is the default view during the Test stage, and it offers various views by clicking the filters along the top of the pane.
    1. Click F2, or the data icon on the right hand side of the toolbar to open this view.
    2. At the top of the Data pane, select a filter to view information.
    3. Click on an item in the data pane to view its details in the Detail pane.
  5. Custom error pages. 4xx responses are automatically identified as error pages. If your site returns 2xx responses with custom error pages, you must configure AppScan to recognize them. This information is essential in deciding whether tests succeed. Not configuring custom error pages will lead to inaccurate results, both false positive and false negative. Therefore, if you found pages in the previous step containing the word "error" in their response but were not classified as error pages, configure them now.
    1. In the Detail pane, click Show in Browser to verify that it is indeed an error page.
    2. Click Set as Error Page.
      Note: You can also define error pages in Scan Configuration > Error Pages, by clicking the plus (+) icon, and defining a string, regular expression, URL or page.
  6. Filtered URLs. Review the list of requests that were not sent, to verify that it does not contain requests that should have been sent.
    1. In the Data pane, select Filtered URLs view, and verify that the filtered URLs should indeed be filtered, and that they are categorized correctly.
    2. If URLs were incorrectly filtered because of their domain, ("Untested web Server"), add the domain to the scan (Configuration > URLs and Servers > Additional Servers and Domains > +).
    3. If URLs were incorrectly filtered because the Path Limit was reached, consider making one of the following configuration changes:

      - Increase Redundant Path Limit (Configuration > Explore Options > Redundant Path Limit)

      - Adjust default Redundancy Tuning (Parameters and Cookies > Redundancy Tuning Defaults)

      - Adjust Redundancy Tuning for individual parameters

  7. Parameter-based navigation. If all or part of the site sends a single URL, while different parameters control content and structure, refer to Sites that use parameter-based navigation.
  8. Parameters. In the Data pane, review the parameters discovered during the Explore stage.
    1. In the Data pane, select Parameters view, to see all parameters discovered during the Explore stage.
    2. If necessary, update the definitions (Configuration > Parameters and Cookies).
  9. Failed Requests. These are requests whose response status was 4xx ("Error"). Review this list to check whether legitimate requests unexpectedly got error responses.
    1. In the Data pane, select Failed Requests view.
    2. 404 Not Found: Click Show in Browser to verify that the URL doesn't exist.
    3. Timeout or Connection Failed: See if the scan needs a higher timeout (Configuration > Communication and Proxy > Timeout), the site's server or environment needs improving, or whether the connection problems are due to requests being sent simultaneously (Configuration > Communication and Proxy > Number of Threads, reduce the setting to "1"), or by too many requests being sent in a given time (Configuration > Communication and Proxy > RequestRate Limit).
    4. 401 or 407 Authentication Required: This means there are areas of the application that require HTTP authentication (set these in Configuration > Platform Authentication).
    5. Other 4xx statuses: Check whether the site returned an error because the user was not logged in. If necessary, record the login procedure again (Configuration > Login Management).
  10. If, after reviewing the initial Explore results, you feel that coverage is insufficient, refer to the next section for possible configuration changes. See Additional configuration.