Why most AI pilots never reach production

We have been asked to look at a fair number of stalled AI projects. Some were ours to fix, some belonged to someone else. The pattern repeats often enough to be worth writing down.

The pilot almost always works. That is the confusing part. Someone builds a proof of concept, it does something genuinely impressive on a curated set of examples, and the room agrees it should be rolled out. Then it is not. Budget gets renewed once, perhaps twice, and eventually the whole thing is folded quietly into next year.

The failure is rarely the model. It is almost always everything arranged around the model.

1. The pilot was measured on the wrong thing

A demo is judged on whether it is impressive. A production system is judged on whether it is reliable on an ordinary Tuesday, against the messiest input your business can produce. Those are different tests, and passing the first one predicts very little about the second.

Before anyone builds anything, agree on the number that has to move. "Improve efficiency" is not a number. "The time between an invoice arriving and being approved, which currently averages four days" is. If nobody can name it, the pilot has no way to succeed or fail, so it will do neither.

2. Nobody owns the unhappy path

Demos run on clean data. Production gets a scan that is slightly rotated, a supplier who changed their layout last month, and a field left blank for reasons lost to history.

The useful question is not what the system does when it works. It is:

• What happens when the model is not confident?
• Who reviews that output, and how long does the review take?
• What does the reviewer do when the answer is wrong?
• How does the correction find its way back into the system?

A model that is right nine times out of ten is genuinely useful when the tenth case lands in a queue somebody actually works through. The same model with nowhere for that tenth case to go is a liability with good marketing.

3. It was built beside the business rather than inside it

Most pilots run on an export. Someone pulls a spreadsheet, the system does its work, and the results come back as another spreadsheet. Nothing about that resembles production.

The integration is usually where the real cost sits: authentication, permissions, rate limits, audit trails, and the twelve-year-old system that will only accept a file drop at two in the morning. None of it is glamorous and all of it is required. If you have not scoped the integration, you have not scoped the project.

4. No one agreed who is accountable when it is wrong

This is the one that stops the most projects, and it is not a technical problem at all.

An automated decision still needs an owner. If the system approves something it should not have, whose name is against it? Until that is settled, deployment keeps sliding, because going live means somebody accepting responsibility they were never formally asked to take on.

Settle it early. It is a governance conversation rather than an engineering one, and it reliably takes several weeks longer than anyone plans for.

How we approach it instead

None of our answers are exotic. They are mostly about sequencing:

• Agree the measurable outcome before any code is written.
• Design the review path in week one, alongside the happy path.
• Scope the integration honestly, including when the number is unwelcome.
• Name the accountable owner before deployment rather than after.

The uncomfortable version

Sometimes the honest answer is that you do not need AI for this. A well-placed rule, a fixed form, or removing a step altogether will often beat a model on cost, speed and explainability. We would rather say that in week one than bill you for six months of finding out.

If you have a pilot that has been nearly ready for a while, the four questions above are a reasonable place to start. If you would like a second opinion on one, tell us about it.