Cognitive

Part 6 · LLM Application Security

LLM01: Prompt Injection

Text a model reads can change what it does, because models have no built-in boundary between instructions and data.

4 min readBuilds on AI Threat Modeling

OWASP LLM01. This material is for defense. Only test systems you own or are authorized to test.

Instructions and data arrive in the same stream of tokens.

What it is

A prompt injection occurs when input alters a model's behavior in ways the developer did not intend. That input can be typed by a user, retrieved from a document, returned by a tool, hidden in an image or audio file, or stored in memory.

The root cause is architectural. Instructions and data reach the model as tokens in one stream, so there is no equivalent of the parameterized query that largely solved SQL injection. Injected text also need not be visible to the person using the application.

How the attack unfolds

An indirect injection against an email assistant. Switch scenarios to see where architectural controls stop the attack.

Figure 30.1. An indirect injection against an email assistant. Switch scenarios to see where architectural controls stop the attack.

Forms of injection

FormWhere the instruction comes from
DirectThe user types text intended to override the application's rules
IndirectWeb pages, emails, documents, code comments, or tool results
MultimodalImages, audio, video, or file metadata
PersistentInstructions saved into memory that influence later sessions
JailbreakPrompts that target the model's safety training specifically

Real-world case

EchoLeak (CVE-2025-32711) affected Microsoft 365 Copilot and was disclosed in June 2025. A single crafted email could trigger data exfiltration without any user interaction. Microsoft fixed it on the server side.

Defenses

Layered controls limit what a successful injection can achieve.

OWASP's position is that no reliable prevention exists, so defenses should be architectural: assume the instruction boundary will eventually be bypassed, and limit what a compromised model can do.

  1. Constrain capabilities. Give the model only the tools and data this task needs.
  2. Break the lethal trifecta. Avoid combining private data, untrusted content, and an outbound channel in one session.
  3. Authorize outside the model. Enforce permissions in deterministic code.
  4. Require approval for high-impact actions. Show the exact recipient, amount, or change.
  5. Control output channels. Block automatic loading of external links and images.
  6. Test adversarially. Measure attack success rates on every change.

Common misconception

Common belief

"A strong system prompt prevents injection."

In practice

A system prompt is more text in the same stream. It can reduce some attacks but cannot enforce a security boundary.

Key takeaways

  • Any content a model reads can attempt to steer it.
  • Current defenses reduce risk but do not eliminate injection.
  • Limit what a successful injection can do.

Explore next