AI is leaving the datacenter. Vision-language models run inside cameras. Robots plan their next move with an on-device LLM. Drones and factory controllers reason about sensor data locally instead of shipping it to the cloud. The industry calls this physical AI: a model on a device, with sensors that perceive the world and tools or actuators that act on it.
Physical AI means the decision happens on the device, not in a datacenter. A security camera runs a vision model that judges, by itself, whether the scene in front of it needs an alert. The humanoid robot in the warehouse plans its grip from what its own cameras see. On the factory line, a quality-control camera rejects a part without asking a server first. No round trip to the cloud sits between perceiving and acting, and that autonomy is exactly what makes these devices useful.
Physical AI devices are still embedded devices. They have firmware, a kernel, a network stack, a supply chain, and, in the EU, a Cyber Resilience Act obligation. Everything we know about securing them still applies. But the model on board adds a class of risk that the classic embedded threat model does not cover. Most teams shipping these devices have not written that risk down yet.
This post opens a series on securing physical AI. It lays out what changes when a model can perceive and act, and gives a threat model you can walk your own device through. The posts that follow will put its defensive principles to work in hands-on case studies.
Why your existing threat model doesn’t cover this
Threat modeling for a traditional embedded device asks familiar questions. What ports are open, what runs as root, what happens when an attacker owns a process, how does an update get trusted. Those questions still matter. Physical AI adds three shifts that break the assumptions behind them. The pattern matches what Microsoft describes for AI applications in general [1], but on a physical device each shift gets sharper.
Every sensor is now an input channel for instructions. A language model treats its whole context as potential instructions. It cannot reliably separate data from commands. On a chatbot, the malicious context arrives in a document. A physical AI device gets its context from the world itself: pixels from a lens, audio from a microphone, text on a package label. Anyone who can put a sign in front of a camera can put text into the model’s context. No packet crosses the network, no file is modified, no CVE is exploited. The attack arrives as light through the lens, and your network threat model never sees it.
Behavior is probabilistic, and there is no patch. A firmware vulnerability is deterministic: found, fixed, versioned, shipped. A model that can be talked into misusing its tools is not a bug you patch. The same input space that makes the model useful makes it steerable, and a rare failure mode becomes a frequent one at fleet scale. You cannot enumerate bad prompts the way you enumerate CVEs. Any mitigation that depends on the model behaving is not a mitigation.
Failure acts on the physical world, unattended. A jailbroken chatbot says something embarrassing. That same jailbreak on a camera exfiltrates footage; on a robot arm, it moves metal. Physical AI devices run headless: no operator reviews the tool calls, and nobody is watching when the model gets convinced to do something else. The blast radius is whatever the device’s credentials, network position, and actuators allow. By default that is far more than the job needs.
Model the loop, not the components
Traditional threat modeling, STRIDE and friends, decomposes a system into components and enumerates threats per component. For agentic systems, that decomposition misses the actual attack shape [2]. An attacker treats the agent as a stateful campaign, where an instruction planted in one place fires an action somewhere else entirely. What you need to model is the loop.
Every physical AI device, whatever its shape, runs some version of the same loop. Each stage of it is an attack surface.
Perception is the injection surface. Sensors turn the world into model input: frames, audio, telemetry, RF. Anything the device can perceive, an attacker can craft. A printed sign, a QR code on a parcel, an adversarial pattern, an ultrasonic command: all of them are inputs the model will read.
Reasoning is the hijack surface. The on-device model interprets input and decides what to do. It follows instructions in its context, and it does not know which parts of the context are yours. Goal hijacking is the default failure mode here.
Action is where compromise becomes consequence. The agent calls tools which result in system operations: file writes, network requests, shell commands, actuator drives. Every tool call uses the device’s own legitimate capabilities. At the system boundary, an exfiltration looks exactly like the camera doing its job.
Memory and state are the persistence surface. Captured data, logs, vector stores, conversation history, and model weights all live on flash. One planted instruction that lands in stored state re-enters the context on every future loop. A tampered model file is a permanent implant.
Connectivity is the propagation surface. The device talks to the cloud, the fleet, and sometimes other agents. One compromised device is a foothold. A shared update channel or an agent-to-agent protocol turns one poisoned input into a fleet problem.
The value of the loop view is that it forces you to trace attacks across stages instead of assessing each stage in isolation. A canonical physical AI attack is exactly such a chain. Crafted input hits perception, the goal gets hijacked in reasoning, and a legitimate tool is misused in action. Nothing in that chain gives a signature or a network filter anything to grab onto.
Start with assets, not attacks
Before enumerating threats, name what an attacker actually wants from your device. For physical AI the asset list is longer than most teams expect.
- Data the device perceives. Footage, audio, and telemetry are often the most sensitive data in the building. All of it is now processed and stored locally.
- Physical actuators. Whatever the device can physically do, an attacker who steers the model can attempt: pan a camera, move an arm, unlock a door, steer a vehicle.
- Credentials and network position. An edge box typically holds cloud credentials and sits on an internal network. The model is a new, softer path to both.
- Model weights and outputs. Weights are intellectual property, and outputs feed downstream decisions. A device whose perception can be silently degraded is compromised even if no data ever leaves it.
- The fleet. Devices ship in thousands. Anything that scales through a shared channel turns one device’s compromise into an incident with a multiplier: updates, configuration, agent-to-agent communication.
Rank these by impact for your device, then walk each one through the loop above. At every stage, ask how an attacker could reach the asset from there. That exercise, done honestly for one afternoon, is the threat model. The output is more than a compliance artifact: it is the list of channels your enforcement has to cover.
Principles for defending it
The threat model points to an uncomfortable conclusion: the model layer cannot be made trustworthy. Prompt injection has no general fix. Guardrails that ask the model to police itself fail against the same steering that caused the problem. The defensive principles follow from accepting that.
Assume the model is compromised, and bound what that buys. Design every control to hold even when the model has been fully hijacked. That means enforcement must live below the model, in the kernel, the OS, or the hardware, where a prompt cannot reach. Default-deny is the posture. Enumerate the few files, hosts, programs, and devices the agent’s job requires, and deny the rest. A denylist loses to the target you forgot. An allowlist makes the forgotten target unreachable by construction.
Give the agent an identity and the minimum mandate. An agent that runs as root, or as a shared service account, hands its full privileges to whoever steers it. The agent needs its own identity: its own user, its own process boundary. Policy can then attach to the agent and to nothing else on the device. Its privileges can be cut to exactly the job: these files, this loopback endpoint, this one actuator.
Know your AI supply chain before it ships. The model weights, the inference runtime, the agent framework: all of it is firmware now. That covers LLM runtimes just as much as the dependency tree around them. It belongs in the SBOM, it carries CVEs, and it deserves the same pre-deployment static analysis as the rest of the image. A poisoned or backdoored model artifact is a supply chain attack your runtime defenses inherit on day one. This is where Exein Analyzer sits in the lifecycle. Scan the image, AI stack included, before it ever reaches a device.
Watch the boundary, because you will not watch the prompts. You cannot log and review every model inference at the edge, and you would learn little if you did. What you can observe is the enforcement boundary. Every blocked exec, every denied connection, and every off-policy file access is a high-signal event. Each one says the agent tried to leave its mandate, regardless of how the model was convinced. Shipped off the device and correlated across the fleet, those events turn one camera’s blocked exfiltration into an early warning for ten thousand others. This is the ground Exein Runtime covers: kernel-level policy enforcement on the device, plus incident detection and attack graphs across the fleet.
Compliance is coming for these devices too. Under the CRA, a physical AI device is a product with digital elements like any other. Risk assessment, SBOM, vulnerability handling, and incident reporting all apply. The threat model you build here is the AI-specific part of the risk assessment the regulation asks for, and it comes at no extra cost.
Where this series goes next
Principles are easy to state and hard to trust until you watch them hold under a real attack. That is what the rest of this series is for. Each post will take one slice of the threat model above, build it on real hardware, attack it, and be honest about what holds and what does not.
Expect hands-on case studies across the whole loop:
- confining an on-device agent with kernel-enforced, default-deny policy
- hardening the perception path against injected instructions
- scanning the AI stack of a firmware image before it ships
- catching a hijacked agent across a fleet
The pattern stays the same throughout: compromise at the model layer, containment at the layers below it.
For kernel-level runtime enforcement on your own hardware, or a firmware analyzer scan contact the Exein team for a demo.
References
[1] Microsoft Security Blog, Threat modeling AI applications: why instruction-following systems break traditional threat modeling.
[2] Christian Schneider, Threat modeling agentic AI: the zone-based lens for tracing multi-step attacks through an agent loop. This post adapts that lens to physical devices.
[3] OWASP, Top 10 for LLM Applications: the standard taxonomy for the model-layer threats referenced throughout.
Author
Thomas Roos, Senior Field Application | Solutions Engineer
