Working with Explore Optimization

This section describes running a scan with Explore Optimization active.

About this task

If your site rewrites parameters into its URLs, or if initial attempts to explore the site, with Explore Optimization disabled, result in very large numbers of URLs or unending scans, the Explore Optimization module can help to reduce the scan to manageable proportions. In addition, activating the Run navigational parameter detection option may help.

Procedure

  1. Configure the Starting URL and any other necessary settings as for a regular scan.
  2. Open Tools > Extensions > Explore Optimization: Configure, and select the Run Explore Optimizer automatically during scans check box.
    Note: If your scan configuration includes Manual Explore data or a multi-step operation, Explore Optimization does not run automatically even if configured, though you can run it manually (Tools > Extensions > Explore Optimization Module > Run).
  3. In the Scan configuration area, optionally change the Run navigational parameter detection setting to True.
  4. Click OK.
  5. Start a scan (Scan > Full Scan).

    If the number of URLs found in the Explore stage (including unexplored URLs) reaches the threshold defined in Minimum links to start module (default: 1000), the Explore stage is paused and the Explore Optimization module starts a phase, consisting of two (main) stages:

    Identify navigational parameters (if configured)
    The module looks for navigational parameters, using the defined Names and Values (configured in Tools > Extensions > Explore Optimization: Configure). If it succeeds in identifying navigational parameters, it then:
    1. Defines them in the list of parameters (Scan Configuration > Parameters and Cookies > main tab)
    2. Sets their Redundancy Tuning to the strictest level
    3. Decreases the default Redundancy Tuning (for non-navigational parameters) to a lower level (see Redundancy tuning defaults)
    Identify URL rewriting
    The module looks for parameters written into the URL. If it finds such parameters, it defines them in the list of custom parameters (Scan Configuration > Parameters and Cookies > Advanced: Custom Parameters tab).
  6. At the end of this phase of Explore Optimization:
    • If any configuration changes have been made, the existing Explore data is cleared and a new Explore stage is run. (If you started the module manually, you are given the option of saving the current data before the new Explore stage is run.)
    • If no configuration changes have been made, a new phase of Explore Optimization is run, using a higher threshold (not user-configurable), in an attempt to gather enough data to identify parameters and reduce the Explore data to a reasonable size.
  7. After the module has successfully run (with one or more phases, and one or more Re-Explore stages), the scan resumes and finishes.
  8. When the scan is complete, review the results, looking for the following indications that it has been successful:
    • The status bar, at the bottom left of the screen, indicates how many tests were created and how many sent. All tests should have been sent.
    • The application tree should be complete, indicating that all the important parts of the site were visited.
    • Review the navigational parameters that were added to see that all important parameters were fully tracked.
    • Review the custom parameters that were added to see that they correctly represent the way the site writes its parameters into its URLs.