OWASP LLM Top 10 Scanner:
Detect Prompt Injection & AI Vulnerabilities
in Your Codebase

An attacker manipulates an LLM through crafted input, causing it to execute unintended instructions or ignore its system prompt. The primary vector: user-controlled data interpolated directly into the system role.

// ❌ VULNERABLE — LLM01 Prompt Injection
const response = await openai.chat.completions.create({
  model: 'gpt-4o',
  messages: [
    {
      role: 'system',
      content: `You are a helpful assistant. Context: ${userMessage}`,
      //                                              ^^^^^^^^^^^^^^
      //  Raw user input in system role = injection attack surface
    },
  ],
});

// ✅ SAFE — Isolate user input in user role
const response = await openai.chat.completions.create({
  model: 'gpt-4o',
  messages: [
    { role: 'system', content: 'You are a helpful assistant.' },
    { role: 'user',   content: userMessage },
    //                ^ user-controlled input stays in user role
  ],
});

LLM responses are passed downstream without validation — into HTML renderers, database writes, or function calls. A compromised or injected model can use this to execute XSS, SQL injection, or SSRF.

// ❌ VULNERABLE — LLM02 Insecure Output Handling
// LLM response injected directly into DOM — XSS vector
const reply = await openai.chat.completions.create({ ... });
return (
  <div dangerouslySetInnerHTML={{ __html: reply.choices[0].message.content }} />
);

// ✅ SAFE — Sanitize before DOM injection
import DOMPurify from 'isomorphic-dompurify';

const raw   = reply.choices[0].message.content;
const clean = DOMPurify.sanitize(raw);
return <div dangerouslySetInnerHTML={{ __html: clean }} />;

Malicious data injected into training or fine-tuning pipelines introduces backdoors or biases. Custodia checks for data-source validation gates, access controls on training datasets, and logging of fine-tune jobs.

Attackers send resource-exhausting inputs: unbounded context windows, recursive prompts, or flooding completions endpoints. Without token caps and rate limiting, LLM API costs can spike to five figures overnight.

// ❌ VULNERABLE — LLM04 Model DoS
// No token cap — adversarial input can exhaust quota
const response = await openai.chat.completions.create({
  model: 'gpt-4o',
  messages: [{ role: 'user', content: userInput }],
  // missing: max_tokens, no rate limiting middleware
});

// ✅ SAFE — Token cap + rate limiting
import rateLimit from 'express-rate-limit';

// Rate limit: 10 requests/minute per IP
app.use('/api/chat', rateLimit({ windowMs: 60_000, max: 10 }));

const response = await openai.chat.completions.create({
  model: 'gpt-4o',
  messages: [...],
  max_tokens: 1000,  // hard cap per request
});

Vulnerable or malicious third-party LLM packages, plugins, and model providers. Custodia checks your LLM SDK versions against known CVEs and flags outdated dependencies.

PII, secrets, or proprietary data sent to external LLM APIs. A GDPR and data governance risk — your users' data leaves your perimeter and enters a third-party model's context window.

// ❌ VULNERABLE — LLM06 Sensitive Information Disclosure
// PII sent to external LLM API — GDPR Article 44 risk
const prompt = `Summarize this customer record:
  Email: ${user.email}
  SSN:   ${user.ssn}
  Card:  ${user.creditCard}`;

// ✅ SAFE — Strip PII before inference
const prompt = `Summarize this customer record:
  ID:     ${user.id}
  Tier:   ${user.plan}
  Joined: ${user.createdAt}`;
  // Only non-sensitive identifiers sent to LLM

LLM-callable tools (function calling / tool use) lack input validation and sandboxing. A single malformed tool call can expose internal APIs or execute unintended operations.

The LLM is granted more capability than necessary: write access to production systems, ability to delete data, or authority to send external communications — all without human approval gates.

// ❌ VULNERABLE — LLM08 Excessive Agency
// LLM can delete database with no human checkpoint
const tools = [{
  type: 'function',
  function: {
    name: 'delete_all_user_data',
    description: 'Permanently delete all user records',
    // No human approval step defined
  }
}];

// ✅ SAFE — Human approval gate before destructive actions
async function delete_all_user_data() {
  const approved = await requestHumanApproval({
    action: 'PERMANENT: Delete all user data',
    severity: 'CRITICAL',
    timeoutMs: 30_000,
  });
  if (!approved) throw new Error('Rejected by human oversight layer');
  // proceed only after explicit human sign-off
}

Systems or users trust LLM outputs without verification gates, leading to automated actions based on hallucinated or incorrect information. Custodia checks for human review checkpoints on high-consequence LLM decisions.

Unauthorized access to proprietary model weights, fine-tuning data, or system prompts via API abuse. Custodia checks for exposed model endpoints and missing authentication on inference APIs.

One Command.
Full OWASP LLM Top 10 Coverage.

Drop custodia scan . in your project root. Custodia auto-detects your LLM stack and runs all 10 checks in under 90 seconds.

$ custodia scan ./src

  ┌──────────────────────────────────────────────────────┐
  │  CUSTODIA.DEV  //  AI SECURITY POSTURE ANALYSIS      │
  └──────────────────────────────────────────────────────┘

  [TRIAGE]   Classifying 847 files...
  [TRIAGE]   Stack detected: Node.js + TypeScript + React
  [TRIAGE]   LLM usage detected: openai@4.28.0, langchain
  [AI AGENT] Activating OWASP LLM Top 10 scanner...

  ── AI SECURITY SCAN ──────────────────────────────────

  [LLM01] CRITICAL  Prompt Injection
          src/api/chat/route.ts:42
          Raw user input interpolated into system prompt
          → AI fix prompt ready in dashboard

  [LLM02] HIGH      Insecure Output Handling
          src/components/ChatBubble.tsx:18
          LLM response passed to dangerouslySetInnerHTML

  [LLM04] MEDIUM    Model Denial of Service
          src/lib/openai.ts:31
          No max_tokens cap on completions request

  [LLM06] HIGH      Sensitive Information Disclosure
          src/api/summarize/route.ts:22
          user.email included in LLM inference prompt

  [LLM08] HIGH      Excessive Agency
          src/agents/executor.ts:87
          delete_database() callable without human approval

  ───────────────────────────────────────────────────────
  AI SECURITY SCORE:   54/100
  OWASP LLM FINDINGS:  5  (1 critical · 3 high · 1 medium)
  FRAMEWORKS MAPPED:   OWASP LLM Top 10 · NIST AI RMF
  PDF REPORT:          custodia.dev/reports/xK9mP2
  ───────────────────────────────────────────────────────

Quota note: Free tier includes 3 scan credits + 5 diff scans/month. The AI Security Agent (OWASP LLM Top 10) runs on Pro ($39/mo) and above. If your codebase has no LLM usage, the AI agent is skipped entirely — no quota consumed, no extra latency.

What is your
AI Security Score?

Free — 3 scan credits against OWASP Top 10 + AI basics. Pro ($39/mo) unlocks OWASP LLM Top 10, NIST AI RMF, and EU AI Act. See pricing →