Five ways an AI automation project goes wrong, from someone who builds them

The failures are boring, and they are almost never the model

When one of these projects dies, the post-mortem is rarely "the AI wasn’t good enough". It is that the thing being automated was a mess, or that nobody was watching it, or that the output had nowhere to go.

Below are the five we run into most. We are writing them down because a skeptical reader deserves the failure list before the feature list, and because recognizing your own situation here will save you more money than any build we could sell you.

One: automating a process that was already broken

Automation is a multiplier. Point it at a process that works and you get more of a thing that works. Point it at one that is muddled and you get the muddle, faster, and now with a technology to blame.

The usual shape is a business whose intake is different every time depending on who answers the phone. There is no agreed set of questions, no agreed definition of a qualified inquiry, and no agreed next step. Asking for an automation of that is asking us to encode a disagreement.

The fix is not exciting. Sit down and write the process as it should run, on one page. Half the value of the discovery hour we do is that it forces this, and some clients get most of what they wanted from that page alone.

Two: no human in the loop

The pitch that sells is "it just runs". The build that survives is the one where a person is still reading. Those are not the same thing and the gap between them is where the horror stories live.

Our default is draft-and-wait. Anything that sends, books or spends is drafted first and released by a person, and it only moves to running on its own after somebody has watched it be right on real cases. Even then, transcripts and logs get read on a rhythm, and one click stops it.

The failure is not usually one dramatic wrong answer. It is drift. The pricing changed and nobody updated the answers. A form field got renamed and half the records stopped populating. Nobody noticed for six weeks because nobody was looking. Build the looking in, or the build has a shelf life.

Three: letting the assistant improvise

A language model will produce an answer to almost anything you ask it. That is the feature and it is also the risk, because an answer produced with no source behind it is a guess delivered in a confident voice.

Anything client-facing should answer from your own material — your pages, your documents, your written answers. Outside that boundary the correct behavior is to stop and hand to a person. We would rather it say "let me get someone" a dozen times a week than invent one thing that sounds authoritative and is wrong.

For regulated practices the boundary is not optional. A law firm’s assistant does not opine on the merits of a matter or on a deadline, full stop, and that restriction goes in writing before anything switches on.

• Client-facing answers come from your own documents, not general knowledge
• Outside the boundary, it stops and escalates rather than improvising
• What it may never discuss is written down during the build
• Refusing is treated as correct behavior, not a bug to be tuned out

Four: no CRM underneath, so there is nowhere for anything to land

This is the most common structural mistake and it is invisible in a demo. The demo shows the conversation. It does not show what happened to the information afterwards, because in a demo nothing did.

In the real deployment, every call, text and form has to become a record with a person’s name on it, a status, and a next action. Without that, your automation is generating output into a void. Calls get answered beautifully and then nothing happens, which is precisely the problem you were trying to solve.

When someone comes to us for an AI receptionist and has no CRM, building the place for the results to land is usually the first piece of work. Not as an upsell. Because the receptionist does not function without it.

Five: buying an assistant when the real problem is that nobody follows up

This one is worth saying bluntly, because it is the most expensive mistake on the list. Plenty of businesses do not have a lead capture problem. They have a lead abandonment problem.

The inquiries are already arriving. They are in an inbox, a form, a voicemail, a stack of notes. Somebody meant to call them back on Tuesday. Adding an AI receptionist to that business increases the number of inquiries arriving into a system that already fails to act on the ones it has.

The tell is easy to check without buying anything. Take the last thirty inquiries from any source and mark each one: contacted within an hour, contacted eventually, or never contacted. If the third column has anything in it, fix that before you spend money on volume.

1. List the last thirty inquiries from every source
2. Mark each contacted within an hour, contacted late, or never contacted
3. Fix whatever produced the third column, with people and a process
4. Only then look at automation, and you will know exactly what to point it at

The sixth one, which is really the first five wearing a suit: buying the demo

Every one of these failures survives a demo, because a demo is a controlled conversation with a friendly participant and no downstream consequences. Nothing in it shows you what happened to the information afterwards, what the assistant does when a caller is rude or confused, or who was supposed to read the transcript.

The questions that actually separate vendors are unglamorous. Whose account is this built in, and what happens to it if we stop working together. Where does the output land. Who reviews it, and how often. What does it do when it does not know something. How do I turn it off in a hurry, and can a non-technical person find that switch.

If the answers are vague, the build will be vague. A vendor who cannot say where the records go has not thought about the part of the project that determines whether it is worth anything.

• Whose accounts is it built in, and who keeps them at the end
• Where does every call, text and form actually land
• Who reads the transcripts, and on what rhythm
• What is the behavior when it does not know the answer
• Where is the off switch, and can anyone in the office find it

We turn work down over this

On a real share of discovery calls, the honest answer is "you don’t need this yet", and that is what we say. Sometimes it is because the process is not written down. Sometimes it is because the volume is not there and a person can still handle it. Sometimes it is because the money is better spent on the thing that brings inquiries in at all.

We are a one-person shop, which cuts both ways. There is no sales team with a quota, so there is nothing pushing a build that should not happen. There is also only so much capacity, so taking on a project that will fail is a genuinely bad trade for us as well as for you.

The discovery output is yours either way — the map of your week and the shortlist of what is worth automating. If you take that and build it yourself, or take it to somebody else, that is a fine outcome.

What a project that works looks like instead

It starts small enough to be judged. One task, with a before and after you can actually see. It runs alongside the human process for a while rather than replacing it on day one. Somebody owns it and reads what it did.

It is built in your accounts, on your number and your calendar and your card, so that if we part ways the automation and everything it produced stay with you. And it has an off switch that a non-technical person can find.

None of that is thrilling. It is also the difference between a system that is still running next year and a subscription somebody cancels in March.