The four ways AI coding goes wrong

Most people’s experience of AI coding tools is a coin flip. Sometimes it produces something useful, sometimes it produces something you throw away, and there is no obvious pattern to which you get.

There is a pattern. Once you can name the four failure modes, you stop debugging the output and start debugging the brief, and the brief is the part you control.

One: the plausible wrong thing

The most expensive failure, and the one that makes people give up.

You get back something that runs. It looks like what you asked for. The variable names are sensible, the structure is conventional, and it is wrong in a way you will not notice until it is in front of a user. Not broken: wrong. It handles the case you described and quietly does something arbitrary with the case you did not.

This is not the model being careless. It is the model doing exactly what an extremely fast, completely literal contractor with no memory of yesterday would do: fill the gap with whatever is ordinary, and not flag it, because from where it is sitting there is no gap.

What was missing: edge cases, and a definition of done. You specified the happy path and nothing else, so the happy path is what you got, plus improvisation everywhere else.

The tell: you find yourself saying “well, obviously it should have…”. If it was obvious, it was obvious to you, from context the brief did not contain.

Two: the blast radius

You asked for one change. You got one change, plus a refactor of the thing next to it, plus a new utility file, plus a dependency you have never heard of.

Each individual decision is defensible. Collectively they are now your problem, and reviewing them costs more than the feature was worth. Worse, you often cannot tell which parts were the thing you asked for and which parts came along for the ride.

What was missing: constraints, and specifically a boundary. Not “what to do” but “what not to touch”.

The fix that actually works: name the file. “Change useInvoice in src/hooks/useInvoice.ts. Do not create new files. Do not add dependencies. Do not modify anything under src/api/.” That is four sentences and it eliminates this entire category.

Three: the amnesiac

Session one, you establish how the project works: the conventions, the folder structure, the reason you do a thing the unusual way. Session two, none of that exists. You are re-explaining your own codebase to something that has already worked in it, and you are re-explaining it badly, because you have forgotten which parts you had to explain last time.

This is the failure mode that makes people conclude the tools do not scale beyond a toy project. It is also the most straightforwardly solvable one on the list, which is why it is worth naming.

What was missing: persistent context. A file in the repository that states the conventions, the constraints, and the things that look wrong but are deliberate, so it is read every session instead of remembered by nobody.

The part people get wrong: they write this file once, as documentation, and never update it. It is not documentation. It is the brief that applies to every task, and it should change whenever you find yourself explaining something for the second time.

Four: the runaway

You asked a question. It started building. Twenty minutes later there is an architecture.

Sometimes the architecture is good, which is what makes this one insidious: it feels like productivity right up until you realise you have committed to a set of decisions you never actually made, on a question you were still thinking about.

What was missing: scope, and a stopping point. There is a real difference between “how would you approach this” and “do this”, and if the brief does not distinguish them, the second is assumed. Execution is cheap now; that is exactly why the decision to execute has to be explicit.

The pattern underneath all four

Every one of these is the same failure wearing a different coat: something was in your head and not in the brief.

That is genuinely good news, because it makes the problem tractable. You are not tuning a model or hunting for a magic phrase. You are writing a specification, discovering it was incomplete, and completing it, which is a skill with a learning curve you can actually climb, and one that transfers to every tool that comes after this one.

It is also, if you have ever run a team, a skill you already have. You have written briefs for humans for years. The only thing that has changed is that the reader stopped filling in your gaps for free, and started showing you where they were.

The thirty-second version

When the output is wrong, do not rewrite the prompt. Work out which of the four it was:

• Plausible but wrong → you left out an edge case or a definition of done.
• Too much changed → you left out a boundary.
• It forgot everything → you have no persistent context file.
• It built before you decided → you did not separate thinking from doing.

Then fix the brief, not the output. The output is a symptom.