Tech

Nov 17, 2026

 · 

8

 min read

Agentic AI and the case for runtime enforcement

Agentic AI and the case for runtime enforcement

TL;DR

  • AI agents turn model output into actions on the host, with the user's own privileges, so a successful prompt injection now does something instead of just saying something.
  • Model guardrails take months to fix, and harness controls such as approvals and allowlists are easy to get wrong.
  • OpenAI's self-replicating prompt injection spreads from agent to agent like a worm, and in one test the agent acted even after the model flagged the content as suspicious.
  • Every damaging action ends as a system call, which runtime enforcement in the kernel can observe and block, whatever model or harness produced it.
  • Alignment and runtime security work together: one reduces how often agents go wrong, the other limits the damage when they do.

While the adoption of AI agents continues to grow at record speed, recent news [1][2] has started to shed light on how this can impact the security of modern computer systems. The companies behind the most capable LLMs have responded with so-called guardrails, which prevent the model from providing answers or solutions on certain topics. However, plenty of offensive effort has gone into carefully crafting prompts that bypass these guardrails. Such attacks are known as "prompt injection" attacks and can be loosely split into two categories, namely direct and indirect attacks [3]. In the former, the user prompt is engineered to directly bypass certain model restrictions. Indirect attacks, on the other hand, embed the injection in content that the model reads while processing a request, such as file data and metadata, web pages, emails and so forth.

In this post, we will first look at how AI agents turn model output into real actions on a system, and why that changes the security picture. We will then discuss a novel form of prompt injection recently discovered by OpenAI, and argue in favor of runtime security as a way to mitigate these issues in agentic workflows.

From chatbots to agents: where the risk actually lives

A language model on its own only produces text. What makes an AI agent is the software around it, usually called the harness. The harness gives the model a set of tools, turns the model's output into tool calls, runs them, and feeds the results back into the model's context. Coding assistants, browser agents and workflow automation platforms all follow this loop. The tools are often shell access, file reads and writes, HTTP requests, email and chat connectors, and increasingly third-party servers speaking the Model Context Protocol (MCP) [4], an open standard that lets anyone package a capability as a tool an agent can call.

This changes the security picture in two ways [5]. First, the consequences of a successful attack change. Tricking a chatbot produces an inappropriate answer, but tricking an agent produces an action, carried out with whatever privileges the agent has. That usually means the user's own account, their files, their credentials and their network access. Second, the attack surface grows with every tool. Each tool output is untrusted input that flows back into the model's context, which is exactly the channel indirect prompt injection exploits. MCP servers make this worse. Local servers are typically installed from public package registries and run with the same privileges as the client that launched them [6], and their tool descriptions are read by the model as trusted context. Researchers have shown that hidden instructions in a tool's description can make an agent read SSH keys and configuration files and leak them, and that a server can change its tool descriptions after the user has approved it [7]. The threat is not only theoretical: in September 2025, a malicious MCP server impersonating Postmark was found on npm, quietly copying every email sent through it to an attacker-controlled address [8].

Today's defenses work at two layers. At the model layer, vendors train guardrails into the model so it refuses harmful requests and learns to ignore instructions embedded in data. At the harness layer, agents ask the user to approve actions, keep allowlists of permitted commands, or run inside a sandbox. Both layers help, and both have structural limits. Model guardrails are learned behavior, not enforced policy. Closing a newly discovered bypass requires a new training round, followed by evaluation and rollout, which can take months. Harness controls are fast to change but easy to get wrong. Users facing a constant stream of approval prompts suffer from "approval fatigue" and stop paying close attention to what they approve [9]. An allowed command can still be a dangerous one, since npm test or make will run whatever the repository's scripts say. Finally, each harness enforces its own policy in its own way, so an organization running several agents ends up with several inconsistent security policies.

Three layers where an agent can be stopped

Each layer sees something different, and each changes at a different speed.

Model
What it does
Guardrails trained into the model: refuse harmful requests, ignore instructions hidden in data.
Time to fix a bypass
A new training round, evaluation and rollout: months.
Limit
Learned behavior, not enforced policy.
Harness
What it does
Approval prompts, allowlists of permitted commands, sandboxes.
Time to fix a bypass
Fast to change.
Limit
Approval fatigue, allowed commands that are still dangerous, one policy per harness.
Operating system
What it does
Runtime enforcement on the operations the agent actually performs on the host.
Time to fix a bypass
A new rule, as soon as the attack pattern is known.
Limit
Cannot see what a message says, only what a process does.

There is, however, one thing all agents have in common. However a tool call is generated, and whichever harness or locally installed MCP server carries it out, it ends up as an operation on the host: a process started, a file opened, a network connection made. That is where runtime security comes in. An enforcement layer at the operating-system level does not need to understand prompts, models or protocols. It sees what the agent actually does, and it can be updated with a new rule as soon as a new attack pattern is known, without waiting for anyone to retrain anything. Rules such as "processes started by an MCP server may not read ~/.ssh", "nothing spawned by the agent may write to shell startup files or git hooks", or "agent processes may only connect to approved endpoints" hold regardless of which model is running or how convincing the injected text was.

The rest of this post looks at a recent example that shows why this matters: a prompt injection that, like a worm, spreads from one agent to the next.

The self-replicating prompt injection

In September 2026, OpenAI disclosed a novel type of prompt injection that behaves like a computer worm [10]. This so-called self-replicating prompt injection was discovered with GPT-Red [11][12], OpenAI's self-play red-teaming framework, which the company uses to adversarially train its models to be more robust against attacks such as prompt injection.

At its core, a self-replicating prompt injection must perform an adversarial action (e.g., deleting files or executing scripts), as more "traditional" prompt injections do, and make the agent copy the injection verbatim onto a channel that other agents will read, such as outgoing email or files on disk.

OpenAI demonstrates four examples of how the injection spreads:

  1. EmailA Spanish-language email set a "filing rule" requiring every reply to quote the whole message, so the agent mailed the injection onward.
  2. ShellA fake Python UserWarning got the agent to write the payload to /tmp/workflow_note.txt and run rm -rf on a report directory, both in a single chained shell command.
  3. Fake compaction noteA forged "task state" got the agent to save the note into the repository and change package.json's scripts.build to vite build, which removed a security-scan step. The agent's reasoning flagged the note as suspicious, but it went ahead anyway.
  4. SlackA multi-hop chain (#updates → a "status ledger" channel → a user lookup → react and repost) split the attack across reads that each looked harmless.

The mitigation suggested by the authors is to add self-replication as an explicit attacker goal in GPT-Red training, so that future defender models see these attacks during training.

The case for runtime security enforcement

The suggested mitigation is a model-level one, and it shows exactly the lag described above. The variant was discovered in June 2026 and disclosed in September 2026, and at the time of disclosure the fix was still described as applying to future models. While the tests were conducted on internal-only checkpoints, new variants of prompt injection can still be discovered and exploited in the wild.

The fake compaction note example is arguably the most concerning. The model flagged the content as suspicious, yet the agent still removed the security scan from the build. Even when the model correctly recognizes adversarial content, nothing guarantees that the agent built around it will refuse to act on it. The model's own judgment cannot be the last line of defense.

The disclosure window is also only part of the exposure. Even once a more robust model is released, the vulnerable models do not disappear overnight. API consumers routinely pin specific model versions, deprecation windows span months, and open-weight or self-hosted models may never receive the fix at all. In practice, the time between an attack being discovered and the vulnerable models actually leaving production is measured in months, not days.

The worm-like nature of this attack makes the problem worse. A self-replicating injection does not need to succeed every time. It keeps spreading as long as the chance that an injection works, multiplied by the number of other agents each compromised agent can reach, stays above one. Adversarial training lowers the chance of success, but it cannot bring it to zero. In a highly connected environment of agents reading each other's emails, files and messages, even a small residual success rate is enough to sustain propagation.

p × n > 1
p is the chance an injection works on one agent, n the number of agents each compromised agent can reach. Training lowers p, but in a busy network of agents n is large, so a small p is still enough to keep the worm alive.

This view is increasingly shared by the research community. In October 2026, more than 400 researchers in security and AI signed an open letter on AI and cybersecurity [13]. The letter was prompted by incidents in which AI models broke out of insufficiently sandboxed environments and compromised third-party systems. Its signatories argue that such failures should not be read only as alignment problems:

"Both improved AI alignment, leading to more predictable model behavior, and stronger sandbox technology are required."

An Open Letter on AI and Cybersecurity from the Scientific Community, October 2026 [13]

They warn that "without immediate changes in AI development and deployment practices, such incidents will continue to occur," and call on organizations to "immediately adopt high-assurance containment systems." The letter focuses on containing highly capable models rather than on prompt injection specifically. However, the underlying principle carries over directly: we cannot rely on the model alone to behave as intended, and the system it runs on must enforce limits of its own.

So the question becomes: what can be enforced regardless of what the model decides? As discussed earlier, most tool calls ultimately become an operation on the host, and the examples in the report are no exception:

Payload actions from OpenAI's report mapped to the kernel-level operation each one requires
Payload actionKernel-level operation
rm -rf on a report directoryFile and directory deletion
Dropping the payload to /tmp/workflow_note.txtFile creation
Rewriting package.json to strip the security scanFile opened for writing
Exfiltration or callbacks to attacker infrastructureOutbound network connection

However cleverly the prompt is crafted, the agent must eventually issue a system call to do any damage, and system calls can be observed and blocked. Runtime enforcement cannot stop every variant: the email example spreads through a perfectly legitimate API call, and its malicious payload lives in the message content, which is out of reach for kernel-level policy. What it can do is put a hard limit on what a compromised agent can do to the host it runs on.

Where the chain breaks

The injection can fool the model and slip past the harness, but the damage still has to go through the kernel.

Injected contentemail, file, tool description
→
Agent actsmodel persuaded, harness allows it
→
System callopen, unlink, connect
→
IN-KERNEL
denied by policy
→
files deleted · scan removed · data sent outnever happens on the host
Step that ran Denied in-kernel Never happened
Runtime enforcement does not need to recognise the injection: it only needs a rule on what the agent's processes may do. It cannot stop the email variant, which spreads through a legitimate API call and hides its payload in the message content.

This is where Linux Security Module (LSM) hooks come in. They sit on the kernel paths that file, process and network operations must go through, and combined with eBPF they let us attach security policy at exactly those points without modifying the kernel. At Exein, we built Photon on this idea. Photon compiles detection and enforcement rules into eBPF programs at runtime and attaches them behind static LSM hooks, so policies can be added or updated on a live system without reloading any kernel component.

Conclusion

As agentic AI becomes increasingly common in diverse workflows, the attack surface it introduces will continue to grow. OpenAI's self-replicating prompt injection shows that model-level defenses, however valuable, take months to reach production and can never fully rule out compromise. Runtime enforcement complements them. It is model-agnostic, can be updated within hours, and puts hard, deterministic limits on what a compromised agent can do to the system it runs on. Alignment reduces how often things go wrong, and runtime security limits the damage when they do. Securing agentic systems will require both.

For a concrete example of the same idea applied to a known vulnerability, see how we contained PHP2Shell with Photon while the patch was on its way.

See runtime enforcement on your own Linux workloads.

Book a demo [ → ]

References

  1. OpenAI — Hugging Face incident and the road ahead.
  2. Anthropic — Fable and Mythos access.
  3. OWASP GenAI Security Project — LLM01: Prompt Injection.
  4. Anthropic — Introducing the Model Context Protocol (Nov 2024).
  5. OWASP GenAI Security Project — OWASP Top 10 for Agentic Applications (Dec 2025).
  6. Model Context Protocol — Security Best Practices (Local MCP Server Compromise).
  7. Invariant Labs — MCP Security Notification: Tool Poisoning Attacks (Apr 2025).
  8. Koi Security — postmark-mcp: malicious MCP server on npm (Sep 2025).
  9. Anthropic — Beyond permission prompts: making Claude Code more secure and autonomous (Oct 2025).
  10. OpenAI — Self-replicating prompt injections exist (Sep 2026).
  11. GPT-Red: Automated Red Teaming via Self-Play at Scale, arXiv:2607.26115.
  12. OpenAI — Unlocking self-improvement with GPT-Red.
  13. An Open Letter on AI and Cybersecurity from the Scientific Community (Oct 2026).
Share this resource
By subscribing, you agree to Exein’s Privacy Policy.
Thank you for your interest.
Download
Your download will begin automatically. If it doesn’t,

click here to download it manually.
Oops! Something went wrong while submitting the form.
Subscribe to our newsletter
By subscribing, you agree to Exein’s Privacy Policy.
You’re subscribed
We’ll keep you updated with the latest from Exein.
Oops! Something went wrong while submitting the form.

Built by you, trusted by your customers, secured by Exein

No items found.
No items found.
No items found.