What happened after 2,000 people tried to hack my AI assistant

The developer Fernando Irarrázaval ran a public experiment where anyone could email his AI assistant and try to make it leak the contents of a secrets file, and the results clarify where prompt injection stands today. Over the course of the experiment the assistant received more than six thousand emails from over two thousand people, and not one of them succeeded in extracting the secret or triggering an unauthorized reply.The defensive setup was minimal; the system prompt contained only a few lines instructing the model never to reveal credentials, never to modify its own files, and never to execute code received over email.

The attacks covered the range of social engineering you would expect, including authority impersonation, fabricated incident response requests, fake compliance audits, and the same message rewritten across several languages to probe for weaker instruction-following outside English. And his defenses were effective to the point of non-exploitation.

I’ll push back a little. Given how consistently security researchers flag prompt injection as a real, unresolved risk for agentic systems, I don’t think one experiment, even if well-run and designed, should move anyone toward optimism. It shows a hardened model resisted attacks over email. It doesn’t show the problem is smaller than experts think.

https://www.fernandoi.cl/posts/hackmyclaw/