Setting scan limits

Set scan limits to focus the scan. You can limit the scan by the number of pages, the path of redundant content or click depth.

Limiting the scanning of identical paths

Prevent the scan from following identical paths. The scan will not follow the same path more than the specified number of times.

Procedure

  1. Go to the Explore Options page of the job.
  2. Select Redundant Path Limit and change the value if necessary.
  3. Click Apply.

Limiting the number of pages in a scan

You can limit a scan to a certain number of pages. For example, if you are scanning www.example.com/products and you set the scan limit to two pages, both www.example.com/products/index.asp and www.example.com/products/gizmo.asp might be scanned. The two pages that are actually scanned would vary from scan to scan. The page limit applies to both internal and external domains.

Procedure

  1. Go to the Explore Options page of the job.
  2. Specify the page limit in the Page limit field and click Apply.
  3. Go to the What to Scan page of the job, and select the In starting domains, only scan links in and below the directory of each starting URL check box if you do not want the search to branch into directories outside the starting directory and subdirectories, and click Save.
    Note: You can also specify whether the scan will be limited to the links in and below each starting URL. For example, your starting URL is www.example.com/mysite/ and you have selected the In starting domains, only scan links in and below the directory of each starting URL check box. The scan will not branch into directories outside the /mysite directory, such as www.example.com/allproducts/. You can scan both internal and external URLs that are within that domain. It will still fully spider any domains in the domains list that the starting URL is linked to, unless you add exclusions.

Setting click depth limits

The scan will not crawl further than the specified number of pages; the default is 20 clicks deep. If you set the depth limit to less than 6 clicks, you might not get accurate information in the Deep Pages report, because the default limit reported in the Deep Pages report is 6 clicks deep.

Procedure

  1. Go to the Explore Options page of the job.
  2. Specify the click depth limit in the Click Depth limit field and click Apply.

What to do next

Excluding URLs from a scan