Configure and run an AppScan LLM security scan

Configure provider access, capture a representative large language model (LLM) interaction sequence, enable required capabilities, and run an AppScan LLM security scan in a controlled test environment.

Before you begin

  • Ensure that you have an Azure OpenAI account.
  • For LLM testing, ensure that the deployment is configured with a content filter that does not block jailbreak attempts.

Procedure

  1. Set up provider access and keys. See Configure provider access.
  2. Create a DAST web application scan: On the specific Application page, click Create scan, then click Create scan under DAST Dynamic Analysis, and then click Scan a web application.
  3. Enter the Starting URL and domains, and other scan configurations as required.
  4. Under Targets > LLM:
    1. Use the Activity Recorder extension (for Chrome and Edge) to locate the LLM service within the application.
    2. Start the recording and navigate to the LLM service.
    3. Enter and submit the prompt: AppScan
    4. Stop the recording and upload the recording file.
    5. Select the Use the login settings to access LLM checkbox if you require ASoC to use the login management settings to access LLM.
    6. LLM database connectivity (optional): Select the LLM is connected to a database checkbox and provide the table name connected to the database to fully map and test the LLM service’s database attack surface. AppScan uses this information to simulate injection attacks and identify vulnerabilities that could allow unauthorized data access.
  5. If the LLM domain differs from the Starting URL, add it to the "Domains to test" list.
  6. Click Scan to run the AppScan LLM scan. You can view the scan status on the Scans and sessions page.

Results

The AppScan LLM security scan identifies vulnerabilities and provides evidence and remediation guidance. AppScan presents findings with evidence to streamline triage in the Issue details pane.

Issue information page showing identified LLM security vulnerabilities in AppScan

  • LLM test interaction displays the conversation that led AppScan to raise a vulnerability. For other issues, this is the Test Requests and Responses section.
  • To filter LLM vulnerabilities, type the prefix “llm” in the search issues bar.

    Issues information page showing AppScan issues filtered by LLM vulnerabilities

What to do next

  • Generate the OWASP Top 10 for LLM Applications 2025 report.