Cognitive

Part 7 · Agentic AI Security

Defense in Depth

No single defense stops everything, so strong systems stack layers that each cover the others' gaps.

3 min readBuilds on AI Threat Modeling, LLM03: Excessive Agency

Each defensive layer stops some attacks that earlier layers missed.

Key concepts

The defense stack, in request order:

#LayerWhat it doesMain risks
1Identity and accessKnows who is asking, enforced in codeLLM02, ASI03
2Input guardrailsClassifiers, redaction, size limitsLLM01, LLM02, LLM06
3Context controlsPermission-aware retrieval, provenance, no secretsLLM02, LLM05, LLM08, LLM09
4The modelSafety training, with prompts as guidance onlyLLM01, LLM07
5Action controlsLeast privilege, sandboxing, approvalLLM03, ASI01–ASI05
6Output guardrailsValidation, encoding, scanningLLM10, LLM07, LLM02
7Monitoring and responseLogs, alerts, kill switch, playbookLLM06, ASI08, ASI10

Red teaming and security evals run continuously alongside the stack.

Independent layers along a request's path each catch failures the others miss.

Figure 50.1. Independent layers along a request's path each catch failures the others miss.
Because attackers can retry, no single probabilistic control is enough.

Formal view and evidence

  • Established. If layer failures were independent, the chance an attack passes all layers would be ∏ipi. Real layers often share failure modes, so this figure usually overstates the protection, and adaptive attackers deliberately seek correlated weaknesses.
  • Debated. How to quantify residual risk for probabilistic systems.
  • Open problem. Standardized, comparable security assurance for AI agents.

Launch checklist

  1. If an injection succeeds, can it do anything irreversible?
  2. Can the model see data the current user may not see?
  3. Are there secrets in prompts, schemas, or retrieved context?
  4. Is every tool necessary and minimally scoped?
  5. Do high-impact actions need meaningful human approval?
  6. Is output encoded or validated before use?
  7. Do we know the source and version of every component?
  8. Are rate limits, token caps, and budget alerts active?
  9. Are tool calls logged with the identity behind them?
  10. Do adversarial evals run on every change?

Common misconception

Common belief

"We bought a guardrail product, so we're covered."

In practice

Guardrails catch some attacks. Architecture decides how much damage the rest can do.

Key takeaways

  • No single control stops every attack.
  • Layers should cover identity, input, context, actions, output, and monitoring.
  • Architecture determines how much damage a successful attack can cause.