OWASP Top 10 for LLM Applications
| # | Vulnerability | Risk | Mitigation |
|---|
| LLM01 | Prompt Injection | Critical | Input validation, output filtering, least privilege |
| LLM02 | Insecure Output Handling | Critical | Output sanitization, content security policies |
| LLM03 | Training Data Poisoning | High | Data provenance, integrity checks, human review |
| LLM04 | Model Denial of Service | High | Rate limiting, token budgets, caching |
| LLM05 | Supply Chain Vulnerabilities | High | SBOM, dependency scanning, model source verification |
| LLM06 | Sensitive Information Disclosure | Critical | Data sanitization, PII filtering, access controls |
| LLM07 | Insecure Plugin Design | High | Input validation, sandboxed execution, auth scoping |
| LLM08 | Excessive Agency | High | Least privilege, human-in-the-loop for high-risk actions |
| LLM09 | Overreliance | Medium | Human oversight, confidence scores, disclaimer requirements |
| LLM10 | Model Theft | Medium | Rate limiting, API auth, watermarking, access controls |
Prompt Injection Deep Dive
| Item | Description |
|---|
Direct Injection | User attaches 'Ignore previous instructions...' to their query. Defense: sandwich prompting, instruction repetition, input sanitization |
Indirect/Stored Injection | Attacker plants malicious content in a webpage or document that an LLM later retrieves. Defense: treat all retrieved content as untrusted |
Multi-modal Injection | Hidden text in images, audio, or video files processed by multi-modal models. Defense: strip metadata, resize/compress all uploaded media |
Jailbreaking Techniques | DAN (Do Anything Now), roleplay scenarios, base64 encoding, language switching, few-shot manipulation. Defense: comprehensive safety training, red teaming |
Token Smuggling | Using special tokens or encoding tricks to bypass filters. Defense: strict token validation, canonicalization before processing |
Recursive Injection | LLM output is fed back as input, allowing injection to compound. Defense: treat LLM outputs as untrusted when re-feeding |
Cross-Model Injection | Output from one LLM poisons another. Defense: sanitize all LLM outputs before passing to downstream systems |
Data Poisoning & Model Theft
| Item | Description |
|---|
Data Poisoning | Adversarial examples added to training/fine-tuning data to create backdoors or degrade performance |
RAG Poisoning | Injecting malicious documents into the retrieval corpus that steer RAG responses |
Fine-tuning Poisoning | Poisoned examples in fine-tuning data can implant backdoors triggered by specific phrases |
Model Inversion | Reconstructing training data from model outputs by querying strategically |
Membership Inference | Determining if specific data was in the training set — GDPR/privacy risk |
Model Extraction | Querying an API thousands of times to train a clone model — IP theft vector |
Watermarking | Embed detectable patterns in model weights/outputs to prove ownership if stolen |
Access Control | API keys, rate limits, and user authentication prevent unauthorized model access and extraction |
Secure Deployment Checklist
| Item | Description |
|---|
API Gateway | Route all LLM API calls through an API gateway with authentication, rate limiting, and logging |
Prompt Firewall | Deploy a prompt firewall (e.g., Guardrails, NeMo Guardrails) that validates all prompts before reaching the LLM |
Output Scanning | Scan all LLM outputs for PII, secrets, injection payloads, and policy violations before returning to user |
Sandbox Tools | Execute all LLM tool calls in isolated sandboxes (Docker containers, restricted VMs) with timeout and resource limits |
Audit Logging | Log every prompt, response, tool call, and user action — immutable logs for forensic analysis |
Least Privilege | Grant LLM agents the minimum permissions needed for each task — scope API keys to specific endpoints |
Model Signing | Verify model file integrity with cryptographic signatures — prevent supply chain attacks on model weights |
Regular Red Teaming | Conduct adversarial testing at least monthly — automated (Garak, Promptfoo) + manual expert red teaming |
Pro Tip: The OWASP Top 10 for LLM is your security baseline. Start with LLM01 (Prompt Injection) and LLM06 (Sensitive Information Disclosure) — these account for 70%+ of real-world LLM security incidents.