four smarter locks and the same mistake

In one of the systems I've built, there was a single automated check whose only job was to make sure a particular value always matched what it was supposed to represent. Over time, that check got rewritten four separate times, each version smarter than the one before it. Each time, someone found a way past it anyway in a fresh adversarial review, not by beating the check directly, but by moving the thing being checked somewhere the check never looked.

By the fourth time, I stopped asking how to build a better check. I started asking a different question: why did this value need a check at all.

That is the whole story, and it is a small one. I think the lesson underneath it is not small.

the same mistake, four times in a row

The pattern repeated with almost no variation. We would write a check that caught the current way the value could drift out of sync. Someone found a way past it anyway, every time, by relocating the actual problem to a spot the check was not looking at. We would rewrite the check to cover that new spot, confident this version closed the gap. It did not. It just moved the gap somewhere else, and someone found it there too in the next round.

Four rounds of this happened before I stepped back. Each round felt, in the moment, like real progress. Each check really was smarter than the one before it. None of that mattered, because the checks were treating a symptom, and the actual source of the problem was never touched by any of the four fixes.

I think what made this one hard to catch early is that every individual round looked like normal, healthy work. Finding a hole, patching it, and moving on is what I would normally call good maintenance. It only reads as a pattern once you count to four and notice the shape repeating instead of resolving.

saying stop instead of writing a fifth check

After the fourth defeat, I said stop. What I actually said was: "The subject box is already working and has what it needs. Stop wasting my time with a test... Just let the subject be a choice and stop making this so difficult. Drop the guard completely so we can keep moving."

I was not being patient about it. Four rounds of the same failure will do that. Underneath the frustration, though, there was a real decision: no fifth check. The check itself was not the problem to solve anymore.

the fix that actually worked

The fix that ended it was not a better check. It was removing the thing that needed checking in the first place. The value in question stopped being something a person could type in freely, and became a fixed choice from a short list of allowed options. Once there was nothing free-form left to drift, there was nothing left for a check to catch, because there was no longer a way to get it wrong.

That is a different kind of fix than the first four. The first four all assumed the flexibility was worth keeping and tried to police it better. The fifth move gave up on the flexibility instead, and the whole problem went away with it.

It also took less effort than any of the four checks it replaced. Each of those had required someone to think through a new way the value could drift and write logic to catch it. Narrowing the choices down to a fixed list took one decision, made once, and there was nothing left afterward that needed watching.

what four failures were actually telling me

I think the real lesson is this: when a safeguard keeps getting defeated, rewriting it smarter is usually the wrong move, even though it is the obvious one. Four defeats in a row is not a sign you need a fifth, cleverer version. It is a sign the check was never the right fix, and that it was standing in for a design decision nobody had made yet.

The question worth asking after a check fails once, let alone four times, is not "how do I close this gap." It is "why does this gap exist at all, and can I remove the thing that keeps opening it." That question is less satisfying in the moment. It also actually works.

I do not think I would have gotten there after the first defeat, or even the second. It took four rounds of the same failure for me to stop treating each one as an isolated bug and start treating them as one problem showing up four different ways. That is probably the actual takeaway: count the repeats before you trust the fix.