OWASP Benchmark with IAST agent

About this task

The OWASP Benchmark Project is a Java test suite that evaluates software vulnerability detection tools. The AppScan IAST Java Agent and IAST Python agent are fully compliant with OWASP Benchmark.

Run OWASP Benchmark with AppScan IAST Java agent

Procedure

  1. Clone BenchmarkJava and BenchmarkUtils from https://github.com/OWASP-Benchmark.
  2. Open a command prompt, change to the BenchmarkUtils directory, and run mvn install -DskipTests.
  3. In ASoC, start an IAST Java session and download the agent zip as described in Starting an IAST session.
  4. Extract the contents of the zip file.
  5. In the extracted JAR, locate secagent.jar in the jar_deployment folder, and copy it to BenchmarkJava\tools\HCL.
  6. From a command prompt, run runBenchmark_wHCL.bat. Wait until the message '[INFO] Press Ctrl-C to stop the container...' is displayed.
  7. Open another command prompt and run BenchmarkJava\runCrawler.bat.
  8. Run BenchmarkJava\createScorecards.bat.
    The test results are located in: BenchmarkJava\scorecard\Benchmark_v1.2_Scorecard_for_HCL_AppScan_IAST_v{IAST_version} files
    OWASP Benchmark v1.2 result comparison


Run OWASP Benchmark with AppScan IAST Python agent

Before you begin

Download and install the following prerequisites. These installations are required to complete the deployment.
  1. Python 3.12 from https://www.python.org/downloads/windows/.
  2. rust from https://rustup.rs/. This is required for one of the OWASP dependencies.
  3. Java 17 from https://builds.openlogic.com/downloadJDK/openlogic-openjdk/17.0.18+8/openlogic-openjdk-17.0.18+8-windows-x64.msi
  4. Maven from https://maven.apache.org/download.cgi#CurrentMaven. Extract it to C:\Program files and add it to your PATH.

About this task

Use the following procedure in a Windows environment.

Procedure

  1. Install OWASP Benchmark with IAST:
    1. Clone OWASP BenchmarkPython from https://github.com/OWASP-Benchmark/BenchmarkPython.
    2. Download the IAST Python agent from ASoC.
      1. Extract the downloaded agent into the BenchmarkPython folder.
    3. cd to the BenchmarkPython root directory and run the following commands:
      1. python -m venv venv
      2. venv\Scripts\activate
      3. pip install -q -r requirements.txt
      4. Run the IAST agent installation script: python install_secagent.py
      5. Run the benchmark with the agent:
        1. Before you run the benchmark, delete the existing contents of BenchmarkPython\results.
        2. Run OWASP Benchmark with the Python agent wrapper and the required environment variables: 
          set IAST_LOG_PATH=results\HCL-IAST.hcl && 
set FINDINGS=true && secagent flask --app app.py run --debug --port 8443 --cert=adhoc
    4. You can now access the OWASP BenchmarkPython application at https://127.0.0.1:8443/.
  2. Run the crawler to generate benchmark results:
    1. Clone OWASP BenchmarkUtils from https://github.com/OWASP-Benchmark/BenchmarkUtils.
    2. In a new terminal, cd to the BenchmarkUtil root directory and run the following command: 
      set "JAVA_HOME=C:\Program Files\OpenLogic\jdk-17.0.18.8-hotspot"&& mvn install
    3. cd to the BenchmarkPython root directory and run runCrawler.bat. The crawler explores all pages in the application, which can take some time. When it completes, review the issues in ASoC.
    4. Create a scorecard:
      1. Run the following command: 
        mvn -Djava.awt.headless=true org.owasp:benchmarkutils-maven-plugin:create-scorecard 
-DconfigFile=scoringconfigfile.yaml
      2. The results are available in BenchmarkPython\scorecard.
      3. The visual summary file is: Benchmark_v0.1_Scorecard_for_HCL_AppScan_IAST_v1.0.0.html
    OWASP Benchmark v0.1 result comparison

    Benchmark v0.1 scorecard for HCL AppScan IAST