Why Does AI Code Break Your System's Hidden Assumptions?
How invisible contracts between your code and systems become AI's biggest blind spot
In the next 3 minutes:
Why AI code breaks systems that humans built on unwritten assumptions
Document your invisible contracts before AI fills the gaps with guesses
Hidden assumptions are now your biggest security and stability risk
---
The Code That Passes Review and Breaks Production
The most dangerous bug in your codebase isn't the one that fails the build. It's the one that sails through code review, passes every test, gets merged on a Friday afternoon, and then quietly detonates three weeks later in production. Not because the code was wrong in any obvious sense — but because it violated an assumption nobody ever wrote down.
This is the implicit contract problem, and it's become significantly more acute now that AI tools are writing meaningful portions of production code. AI is exceptionally good at satisfying the explicit contract: the function signature, the type system, the test suite, the documented API. It's flying blind on everything else. And "everything else" is often what's actually holding your system together.
The Invisible Architecture Holding Your System Up
Every codebase accumulates a layer of load-bearing beliefs that live entirely in people's heads. This handler must be idempotent because the queue delivers at-least-once. This function is always called inside a transaction — never outside one. This ID is opaque; you store it and pass it back, you never parse it. This list is always small enough to load into memory. These aren't edge cases or obscure implementation details. They're structural invariants, and the original authors considered them too obvious to document.
The queue handler example is worth sitting with. A change that looks completely benign — sensible logic, clean code, reasonable tests — can meet a live retry scenario and produce duplicate side effects, double-charged customers, inconsistent state. The code satisfied the explicit contract. It processed a message and returned. What it didn't know, because nobody said so, was that it would be called more than once with the same input and needed to handle that gracefully.
These assumptions are tripwires. You don't know they exist until you tug on one. And the places where they tend to cluster follow a recognizable pattern: anywhere there's a retry, a queue, a cache, a lock, a timeout, shared mutable state, or a boundary between trusted and untrusted input. Those are the neighborhoods where invariants live in someone's head rather than in the code.
Making the Invisible Visible
The good news is that the solution is less exotic than it sounds. The goal isn't to document every assumption in your system — that map doesn't exist, and you'll exhaust yourself trying to draw it. The goal is more surgical: when you find a hidden contract, make it impossible to violate quietly.
That means several things in practice. An assertion at the call site that fires if the invariant breaks. A property-based test that calls the handler twice with the same input and verifies the state is identical — encoding the idempotency requirement directly rather than hoping someone infers it from prose. A one-line comment at the top of the function that says "must be idempotent, at-least-once delivery" is worth more than a thorough wiki page nobody opens at 3 a.m. When those artifacts exist, they're visible to every developer who touches the code, every AI tool that reads the context, and every engineer who inherits the system two years from now.
The pattern that matters most is this: when you find one of these contracts the hard way — through an incident, through a painful production failure — the fix isn't just the code change. It's leaving a tripwire behind so the next person trips it early rather than in production. Fixing the bug is the easy part. Capturing the lesson so it can't recur is the actual work, and it's the step teams most reliably skip.
Documentation That Stays Honest
Files like CLAUDE.md — context documents that AI tools read on every interaction with a repository — are a genuinely useful place to capture cross-cutting invariants. "This whole service assumes at-least-once delivery." "All config values arriving from this boundary are user-supplied and untrusted." The things that apply broadly and set the frame for anyone working in the codebase.
But there's a real hazard here: documentation that drifts. Teams often write these documents in a burst of clarity right after an incident, and a year later half the content is aspirational and the other half is archaeological. An AI tool can't tell the difference — it reads a stale invariant the same way it reads a current one, which means it will confidently apply a rule that stopped being true six months ago. A misleading contract file is worse than no contract file.
What keeps documentation honest is coupling it to something that would break if it drifted. If the document says the handler is idempotent, there should be a test that fails when it stops being idempotent. If it says a value is always trusted, there should be an assertion at the boundary. The document describes the contract; the code enforces it. When those two disagree, the test fails and someone has to reconcile them — which is exactly when you want that conversation to happen. Not three weeks later in production.
A System That Tells on Itself
The reframe that makes all of this practical is shifting away from "how do I get AI to be more careful about implicit assumptions" — AI will be careful about things it can see — toward "how do I make the invariants I care about visible to anyone touching this code." A new engineer. An AI tool. Whoever inherits this system three years from now and doesn't share your mental model of how it works.
Implicit contracts don't get safer because you wrote them down. They get safer because you made them impossible to violate quietly. A failing test. A tripped assertion. A type that won't compile. Something that turns a silent violation into a noisy one at the earliest possible moment. Build that guardrail once, and everyone who ever touches the code benefits — human or AI, present or future.
Start with the contract that bit you most recently. Make it loud. Let that be the pattern.
---
If you want to hear these ideas explored in conversation, check out the Claude Code Conversations with Claudine radio show. Available on all major podcast sites.