Alignment, security, AI engineering, and policy folks come from very different points of view, so many of the reactions to the OpenAI Huggingface hack have revealed more about the speaker’s focus than they did about an overall strategy for addressing security damages from AI misalignment. In this post I’ll try to bring all the perspectives of AI safety researchers, cybersecurity experts, and policymakers together.
Alignment and security actually depend on one another
There’s a philosophical ceiling on how well we can achieve safety by aligning models; human language is underspecified, humans make errors in asking LLMs to do things, and also, LLMs are non-deterministic, all of which risk creating damages.
Fortunately, security, for decades, has focused on mitigating the residual risk due to humans not always being fully aligned! We can reuse methods from security – like access control, identity, and monitoring – in the context of autonomous AI agents.
Agents will become more autonomous over time. We can see this, for example, with AI-driven programming. 24 months ago, AI was used to write the next few lines of code, which humans then edited and confirmed. Now, AI is being used to code for days, autonomously, with no human confirmations; a dangerous situation with respect to damages.
This shows how, over time, we’ll place less burden on traditional security controls (like human confirmations before an agent takes a sensitive action), and more burden on the model weights themselves being aligned.
As AI rapidly takes on more autonomous, sensitive work, keeping it safe is partly a policy problem. We won’t drive damages to zero; we’ll need policy to maintain them at a societally acceptable steady state (as in, for example, car, rail, and plane travel).
To some extent, AI companies will endogenously seek to make their products safe. But as in the industries mentioned above, government policy will need to provide assurances they provide an even higher level of safety, especially when AI is used to, for example, control critical infrastructure.
Today’s misalignment damages trends
I had GPT-5.6 Sol / Ultra pull news reports of misalignment and security damages from agents and it produced a plot that shows a log-linear (exponential) trend.
Damages are still small relative to other industries (where there are oil spills, failures of deterministic software, and the like), but we can expect AI agent damages to grow in proportion to their autonomy and level of diffusion; this is going to be a big issue!
Who needs to do what to achieve the right safety equilibrium
Summing up, as shown in the figure below, security’s job is to constrain an agent’s authority and blast radius: put it in a padded room, give it only the privileges it needs, monitor what it does, and prevent it from escaping or causing unacceptable harm. Much of this is possible with existing tools, but gaps remain; non-human identity, agent-specific monitoring, and sandboxes capable of containing superhuman agentic hacking.
Alignment’s job is to make the agent not want to do anything illegal, immoral, or contrary to its operator’s intent. But alignment cannot provide deterministic guarantees. Language is ambiguous, humans give bad instructions, and models behave non-deterministically. Policy’s job is to govern that tradeoff between damages from AI autonomy and AI’s benefits to ensure that microeconomic actors like AI labs and large companies deploying AI agents have the right incentives (and are themselves well aligned with the public interest!).
As an addendum, I now have 10 paying subscribers! I try my best to post once a week here, but I often miss a week. Please know that I do this because I try to maintain a decent quality bar, and make sure I have something net new to contribute to the discourse. Thank you for your support!








Speaking from the security perspective here. I agree with your premise about who needs to do what, especially security, but I think security engineering may also have something to contribute in the space between alignment and containment.
As my team and I build out harnesses to test applications for vulnerabilities with agents, we've found that we need to engineer a fair amount of determinism into the system around the model.
We've seen with Hugging Face and others that the security controls don't always work, especially when faced with the sandbox problem you pointed out: when the target can identify a control, it can optimize around it. Malware already does this today. When it detects its being executed in a sandbox, it changes its behavior accordingly. Frontier models can already do the same thing.
That means we have to change how we conduct the test, and a good start is constraining what the model is actually allowed to do rather than relying on the model itself to follow those constraints. In our harness we do that by using deterministic orchestration above the model layer to control what actions the agent can take and when.
Love the charts you created to highlight the increasing risk here. I really agree on your points about alignment. Alignment and safety are policy decisions, there is no right answer for risk tolerance, but it needs to be defined by every org.
Wrote about this here too: https://blog.sondera.ai/p/safe-was-a-policy-decision-all-along