What reverse-engineering games taught me about building AI

When I was nine, I wanted to know why a game did what it did. Not how to play it better — why the numbers on the screen were the numbers they were, and where they lived in memory. That question turned into a hobby, and the hobby turned into a few years of taking games apart to see how they worked from the inside.

I started with Fortnite and later moved to Valorant. What I built were cheats: software that read the game's state and acted on it in ways the game didn't intend. I'm not going to write a how-to here, and I've left out anything that would work as one. What matters for this essay is the shape of the problem, not the method.

It didn't stay a solo project. It grew into a real operation with actual users, enough of them that an update breaking on a Saturday meant a bad weekend for more people than me. For a while it was one of the more popular options in its corner of the scene. I was somewhere between eleven and thirteen for most of it, which is the part I think about most now.

The other side of the screen

The interesting part was never the game. It was what stood on the other side. Anti-cheat systems (BattlEye, Easy Anti-Cheat, and eventually Riot's Vanguard) are built by professionals whose entire job is to end exactly what I was doing. Vanguard runs at the kernel level, with more access to the machine than almost anything else on it. These systems changed constantly, sometimes weekly. A method that worked on Monday could be dead by Friday because someone on the other side had shipped a patch aimed at it.

I wasn't fighting a game. I was in a slow argument with a team of engineers I'd never meet, and every week one of us was wrong.

To keep up you had to think in two directions at once. You had to model the defender: what they were looking for, what a detection looked like, what they assumed normal behavior was. And you had to model how they were thinking about you — what they expected an attacker to try next, and where that expectation left a gap. Most of the work was reading someone else's system carefully enough to predict it.

The letter

Then Riot's lawyers sent a cease-and-desist. Riot does this for real; they have taken cheat makers to court for millions. Getting a letter like that when you're a teenager is a specific feeling. Up to that point it had been a puzzle with a scoreboard. The letter made it clear that the thing on the other side wasn't a scoreboard. It was a company with a legal department and a decision to make about me.

So I stopped, and not gradually. I'd like to tell you the ethics arrived before the letter did, and some of it had, but the honest version is that I was a kid who had wandered into something with real stakes and only understood the stakes when they were printed on letterhead. I don't recommend the path and I wouldn't take it again. What I won't do is pretend it didn't happen, because the skills were real and so was the lesson.

What it actually taught me

Understanding adversarial systems — how a defender reasons, and how an attacker reasons about the defender — is most of what security is. It turns out to be a large part of frontier AI too:

• Red-teaming. Probing a model with inputs chosen to make it misbehave, before someone less friendly does.
• Robustness. Asking whether performance holds when the inputs are adversarial rather than average.
• Evaluations. Building tests an optimizer can't quietly game, because a metric under pressure stops measuring what you thought it did.

These are the same problem I spent years inside, pointed at targets that matter more than a video game. A system that has never been attacked is one whose weaknesses you simply haven't found yet.

Where it points now

The instinct is the one I had at nine, just aimed at higher stakes. Take the system apart. Find where it breaks. Then build something that still holds when someone is pushing on it on purpose. In Littman, a legal decision system, the adversary usually isn't a person trying to cheat. It's the ordinary way these systems fail: confidently and invisibly, on the one case where being wrong is expensive. So I build them to expose their own reasoning and to fail loudly instead of quietly. The reflex to ask “how would I break this if I wanted to?” is the same one that used to keep me a week ahead of an anti-cheat team. It is just doing honest work now.

One piece of this has a logic worth pulling out on its own: the way any detector starts decaying the moment it ships, because the other side begins routing around it immediately. I wrote about that in the half-life of a detector. It isn't an abstract idea to me. I spent a chunk of my childhood on the wrong side of it, and I learned it the way you learn things that cost you something.