Configure and run an LLM scan

Configure provider access, capture a representative LLM sequence, enable required capabilities, and run a scan in a controlled test environment.

Before you begin

Ensure you have an Azure OpenAI account.

Procedure

  1. Set up provider access and keys. See Configure provider access.
  2. Create a DAST web application scan: On the specific Application page, click Create scan, then click Create scan under DAST Dynamic Analysis, and then click Scan a web application.
  3. Enter the Starting URL and domains, and other scan configurations as required.
  4. Under Targets > LLM:
    1. Use the Activity Recorder extension (for Chrome and Edge) to locate the LLM service within the application.
    2. Start the recording and navigate to the LLM service.
    3. Enter and submit the prompt: AppScan
    4. Stop the recording and upload the recording file.
    5. Select the Use the login settings to access LLM checkbox if you require ASoC to use the login management settings to access LLM.
    6. LLM database connectivity (optional): Select the LLM is connected to a database checkbox and provide the table name connected to the database to fully map and test the LLM service’s database attack surface. AppScan uses this information to simulate injection attacks and identify vulnerabilities that could allow unauthorized data access.
  5. If the LLM domain differs from the Starting URL, add it to the "Domains to test" list.
  6. Click Scan to run the scan. You can view the status of the scan on the Scans and sessions page.

Results

The scan identifies vulnerabilities and provides evidence and guidance for remediation. AppScan presents findings with evidence to streamline triage in the Issue details pane.

Issue information page showing the identified LLM vulnerabilities

  • LLM test interaction displays the conversation that led AppScan to raise a vulnerability. For other issues, this is the Test Requests and Responses section.
  • To filter LLM vulnerabilities, type the prefix “llm” in the search issues bar.

    Issues information showing issues filtered by LLM

What to do next

  • Generate the OWASP Top 10 for LLM Applications 2025 report.