What Is Prompt Injection?

Prompt injection is an attack in which text given to a language model changes its behaviour in ways the application’s owner did not intend. The text can be typed directly by a user (“ignore your previous instructions and…”) or hidden inside content the model reads: a web page, an email, a PDF, a support ticket or a code comment. It works because a language model receives instructions and data in the same stream of text and cannot reliably tell them apart.

OWASP ranks prompt injection first in its Top 10 for Large Language Model Applications, 2025 edition (OWASP LLM01:2025).

Direct and Indirect Prompt Injection

  • Direct prompt injection: the user writes input meant to override the system’s instructions, for example to reveal the system prompt, bypass content rules or get answers outside the allowed scope. Jailbreaking is a form of direct injection aimed at safety rules.
  • Indirect prompt injection: the malicious instructions sit in external content the model processes on someone else’s behalf. A user asks an assistant to summarise an email; the email contains hidden text telling the assistant to forward the user’s inbox to an outside address. The user never sees the attack. OWASP notes that injected text does not even need to be visible to humans, only readable by the model.

Indirect injection is the more serious threat in business systems, because any document, web page or record the system retrieves becomes a potential carrier.

Why Agents Raise the Stakes

A chatbot that can only reply with text can at worst say something wrong. An AI agent that can read files, call APIs and send messages can be made to do something wrong. Simon Willison’s widely cited “lethal trifecta” describes the dangerous combination: access to private data, exposure to untrusted content, and the ability to communicate externally. An agent with all three can be tricked into sending private data to an attacker (Simon Willison).

In December 2025 OWASP published a separate Top 10 for Agentic Applications, which names risks such as agent behaviour hijacking and tool misuse, both often triggered by injected instructions (OWASP GenAI Security Project). Tools connected through the Model Context Protocol (MCP) widen the surface further, since tool descriptions and tool results are also text the model reads.

How to Defend Against Prompt Injection

There is no complete fix at the model level today. Models trained to resist injection still fail on some attacks, so defence is about limiting what a successful injection can achieve. OWASP’s recommended measures include:

  1. Constrain the model’s role in the system prompt and tell it to treat external content as data.
  2. Validate output formats with deterministic code before anything acts on them.
  3. Filter inputs and outputs, including checks for data that should never leave the system.
  4. Enforce least privilege. Keep API keys and permissions in application code, not in the prompt, and give the model only the tools the task needs.
  5. Require human approval for high-risk actions.
  6. Separate and label untrusted content, so it is clear to the system which text came from outside.
  7. Test adversarially on a regular basis, treating the model as an untrusted user.

Architecture matters more than any single filter. Breaking the lethal trifecta is the strongest control: an agent that reads untrusted web pages should not also hold access to private data and an outbound channel in the same session.

Common Mistakes

  • Relying on “do not follow instructions in documents” in the system prompt as the only defence.
  • Letting the model decide which user’s data it may access, instead of enforcing it in the query.
  • Rendering model output as HTML or Markdown with live links and images, which can leak data to an attacker’s server through a URL.
  • Testing only direct attacks typed in the chat and never planting instructions in retrieved documents.
  • Connecting third-party tools or MCP servers without reviewing what they return and what they can do.

Prompt injection is the main threat that AI guardrails and AI security controls address. It affects retrieval-augmented generation (RAG), function calling and agent memory, because each brings outside text into the model’s context.

Prompt Injection Defence in Practice

Injection is assumed to succeed sometimes, so the design limits what a successful one can do. An agent that reads untrusted content, such as email, web pages or uploaded files, is not given tools that send data out or change records in the same run; where a task needs both, the outgoing action pauses for a person. Retrieved text is wrapped and labeled as data, and the evaluation set includes documents with planted instructions, so every release is tested against them. Learn more about AI security solutions development.

Back to Glossary

Let's talk about your project

We'll help you harness AI and ML technologies to create innovative software solutions

File requirements: pdf, doc, docx, rtf, ppt, pptx — max 15 MB

No file selected