Explore Optimization module

This extension helps improve scan efficiency by eliminating unnecessary URLs from the Explore results.

The extension can particularly improve performance in cases where URL rewriting is used by the site, inflating the Explore stage of the scan with duplicate URLs that appear different due only to built-in parameters.

The Explore Optimization module identifies URLs that are rewritten to include parameters. For example, if URLs such as those below occur a large number of times, AppScan would create a folder for each one where in fact these are actually parameters that have been rewritten into the URL for convenience.

http://mysite.com/2010/10/01/
http://mysite.com/2010/10/02/
http://mysite.com/2010/11/01/
http://mysite.com/2010/11/02/
http://mysite.com/2010/12/01/
http://mysite.com/2010/12/02/

A case like this could result in thousands of unnecessary URLs, or a never-ending Explore stage.

When it encounters a large number of URLs that seem related, the module analyzes them. It then creates a custom parameter (in this case it would be /[year]/[month]/[day] ), clears the Explore results, and runs a new Explore stage, which may have far fewer URLs to be tested in the Test stage. This can greatly improve performance.

Note: An additional functionality of this module, that can be useful in maximizing scan efficiency, is Run navigational parameter detection (see Configuring Explore Optimization for details.

Running the module automatically

It is suggested to run the module during scans whenever the number of links discovered in the Explore stage (including unvisited links) reaches the configured threshold. This threshold ("Minimum links to start module"), is generally at least 1,000.
Note: If you start a scan from the command line interface (CLI), Explore Optimization does not run automatically even if configured.
Note: If your scan configuration includes Manual Explore data or a multi-step operation, Explore Optimization does not run automatically even if configured, though you can run it manually (Tools > Extensions > Explore Optimization Module > Run).

To change the configuration:

  1. Click Tools > Extentions > Explore Optimization: Configure.

    Explore Optimization Configuration dialog box opens.

  2. Select the Run Explore Optimizer automatically during scans check box.
  3. Click OK.

Running the module manually

You can run the module manually on any Explore results, even of the number of URLs discovered is less than the configured threshold ("Minimum links to start module"). However the module is most useful when the number of URLs is at least several hundred.

To run Explore Optimization manually:

  • Click Tools > Extentions > Explore Optimization: Run.

    The module begins to Explore, starting at the configured Starting URL. When the Explore stage is complete it analyzes the results. If optimizations are discovered, it will give you the option of saving the first set of results before clearing them and re-Exploring.

See also:

Redundancy tuning

Redundancy tuning defaults