Understanding Test Optimization

How it works

A full regular AppScan Standard scan typically sends thousands of tests and may take hours, in some cases days, to complete. During the early stages of development, or for a quick overall evaluation of the current security posture of your product, you can use Test Optimization to get the results you need in a shorter time frame, by choosing a balance between speed and issue coverage. There are three levels of optimization, and the table below shows some suggested use case for each level.

Our intelligent test filters are based on statistical analysis, and filter out certain tests – or even specific test variants – to produce a shorter scan that identifies the more common, severe and otherwise important vulnerabilities only. AppScan fix packs and ifixes keep you up-to-date with the latest optimization filters. Using Test Optimization can greatly reduce overall scan time when fast results are more important to you than a thorough, in-depth scan.

Test Optimization is applied to whichever Test Policy you select for the scan, so not all tests in the policy are sent. Note that the optimization setting makes no difference to the Explore stage, it is the (much longer) Test stage that can be greatly reduced.

Test optimization is activated from Configuration > Test policy and optimization.

FAQ

Q: Does Test Optimization apply to all Test Policies?

A: Yes. Test Optimization filters Test Policies based on our statistical analysis of test results, that is regularly updated.

Q: Does Test Optimization filter out entire tests?

A: Usually it filters out only specific test variants. If this changes in the future, the change will be documented.

Q: Is there any way for me to know exactly which tests, or variants, were filtered out of the Test Policy I selected?

A: This is not currently possible.

Q: Does Test Optimization change other configuration settings, and can I see these changes in the configuration dialog box?

A: Currently no configuration changes are made. This may happen in future AppScan releases, but if it does the changes made will be indicated.

Q: If it scans faster, why shouldn’t I always use Test Optimization?

A: Test Optimization is great when you need faster results, but it is not as thorough as a non-optimized scan. We recommend optimized scans when speed is important, but that you also back them up with full scans at regular intervals.

Q: Can I expect the results of two optimized scans on the same site to be identical?

A: Since our team is constantly analyzing and updating the settings, each AppScan update has improved optimization settings, and therefore even if the site is unchanged the results may not be identical. However it is unlikely that a test that revealed an issue in the earlier scan would be filtered out of the later scan with the same optimization level.