Parallel processing for Java applications

Java analysis now uses parallel analysis techniques, resulting in significantly faster generation of IRX files. This revised approach to analysis also caches converted class files into the analysis format on the client system, analyzing the stored copy instead of rebuilding it each time analysis is performed.

Parallel processing slices code into smaller, more easily scanable pieces for analysis. These smaller pieces of code can then be scanned at the same time. Not only does this take up less memory and CPU space, results are available faster.

Note: Parallel processing is the default behavior. Use the properties described in this topic to further customize scanning, or turn off parallel processing.

General parallel processing properties

To set the property in the client running appscan prepare, prepend the property with -D.

Note: Command line options used for parallel processing apply to the CLI and the Maven and Gradle plugins.


Property	Values	Description
`USE_SLICER`	true\|false Default: true	When set to `true` enables the use of the parallel processing.
`IRX_MINOR_CACHE_HOME`	`$user.home/.appscan/irx_cache`	Specifies the location of the file cache. Windows mapped and UNC drives or Linux mounts can be used here.
`DISABLE_CACHE`	true Default: not set	Prevents using the cache. This will force a re-run of IR generation process on each class file even if there is a cache entry.

Examples

Run with parallel processing enabled and the file system cache stored within $user.home/.appscan/irx_cache:

CLI
```
$ appscan.sh prepare 
```
Gradle
```
gradle appscan-prepare
```

Maven

mvn com.hcl.security:appscan-maven-plugin:prepare

Run with parallel processing enabled and avoid using the cache:

CLI

$ appscan.sh prepare -DDISABLE_CACHE=true

Gradle

gradle appscan-prepare -DDISABLE_CACHE=true

Maven

mvn com.hcl.security:appscan-maven-plugin:prepare -DDISABLE_CACHE=true

Set the file cache to use the mapped drive s: in the irx_cache directory:

CLI

> appscan.bat prepare -DIRX_MINOR_CACHE_HOME=s:\\irx_cache

Gradle

gradle appscan-prepare -DIRX_MINOR_CACHE_HOME=s:\\irx_cache

Maven

mvn com.hcl.security:appscan-maven-plugin:prepare -DIRX_MINOR_CACHE_HOME=s:\\irx_cache

Run with parallel processing disabled:

CLI
```
$ appscan.sh prepare -DUSE_SLICER=fals
```

Gradle

gradle appscan-prepare -DUSE_SLICER=fals

Maven

mvn com.hcl.security:appscan-maven-plugin:prepare -DUSE_SLICER=fals

Logs

The following logs are generated as part of parallel processing:

scan.manifest : This file is generated as part of the IRX file generation and lists all the items that were discovered. It provides a list of languages and the artifacts that were scanned for each of the languages. Third party artifacts are excluded by default.
scan.mainfest.json: Contains the same information as the scan.manifest file in JSON format for easier parsing.
StaticAnalyzer-Errors.slice.log Provides errors that might have occurred in a slice if the IRX file is not generated successfully. It lists the slice number and the corresponding error message for that slice. If this file is empty, then there are no error messages related to IRX file generation in the slices.
stage.stats.json: Outline the packages being analyzed for Java, the zip format files staged (ear, jar, war, rar) and the memory footprint each of those larger zip format files took to extract.
slice.prepare.stats.json: Contains each class file and its associated bucket along with whether or not it was retrieved from the cache. If not retrieved from the cache, the time it took to generate the slice IPVA file.

Buckets

Each bucket has a folder with number as folder name under the slices directory. Under the logs directory of each slice in the slices/<language>/<bucket_number>/logs directory, there is a scan.manifest file that lists the artifacts and languages that are present in that bucket.

Alternatively the slice.prepare.stats.json file contains bucket association information for each class file being analyzed.

Results

The new program slice mechanism changes some finding signatures to be slightly different than they were before. This may result in a finding being produced that is similar to one already triaged. This happens only the first time program slice analysis is run against an application which already has findings from the previous scanning mechanism. It also may require some findings to be seen as new and require triage as before.