Blog

From AI pilot to production: what a demo does not yet prove

A demo shows that a model can answer. Production only starts once the task, boundaries, exceptions and measures are agreed.

Colleagues looking at a demo on a laptop together

A demo can win people over quickly. You give a language model an order email, it pulls out the item codes and quantities, and the answer looks usable. That is a useful first test. But it says little about whether an employee will really stop retyping that order tomorrow.

Between those two moments lies the work needed to take a pilot into daily use. A real order email has a sender, attachments, a customer number, unusual prices and sometimes an error in the source file. The outcome has to go into an existing system. Someone has to be able to see why an order did or did not go through. And when something is wrong, the email has to reach the right colleague. A demo proves none of these things.

A pilot without a clear brief gets stuck

Sometimes the brief is: see what is possible. That produces a test set-up without a defined task, owner or end date. A team compares models and collects examples. Then the pilot waits for a bigger budget, an architecture decision or the next priority. Meanwhile, the person who works the inbox every day is no better off.

A better start is to name an action rather than choose a model. For example: read standard order emails from a shared mailbox, check the fields against customer and product data and prepare a draft order in the ERP system. That is small enough to test and big enough to take work off an employee's hands.

Setting the boundary is part of the job. Only messages from known senders, with one order each and no price discrepancy, go straight through. An unknown customer, a missing order number or a price difference goes to a queue. Those rules make clear where the software stops and a person decides, without making the solution any less intelligent.

Production needs an owner and a measure

A working integration does not put a process into production. Someone first has to be able to answer, on a Monday morning: which cases went well, which went to a colleague and what do we do if the source application or the model changes?

So before building, pin down four things. Describe where the task starts and ends. Name the system where the outcome lands. Note which cases must never go through automatically. Choose a measure the process owner already recognises, such as the lead time of an order, the number of manual entries or the share of orders corrected afterwards.

That last point guards against a misleading score. A high recognition score can still create a lot of rework when errors quietly end up in the ERP. Recognition is only part of the picture. What counts is the number of cases that reach the right next step without correction.

See one process through to the end

A first project does not need to be a large programme. A week of observation shows how much work comes in, which exceptions are common and where the data comes from. Then you build one process, with the screens, permissions, integrations and checks it needs. Six to ten weeks is a realistic timeframe for a well-defined process, not a promise for every problem.

When capacity is limited, one workflow with the right permissions, exception handling and support is easier to assess than several separate demos. That one solution then gets the attention the final stretch needs: signing in with the right permissions, handling an error, testing a change and naming someone responsible. Once the workflow is stable, it becomes a foundation for the next process.

The value is in repetition

When an employee no longer has to retype a recognisable order, you save more than that one action: the gain returns with every similar message that follows. How much that adds up to depends on the volume and on the time it takes now. That is why measuring the baseline belongs at the start, not in the final presentation.

A pilot that stays a demo has not necessarily failed. Sometimes it shows just in time that the source data is too messy or that the process happens too rarely. In that case, stopping is a good outcome. The mistake is calling the demo a product without deciding what it takes to use it safely every day.

Three ways to start

Want to test an existing pilot against a real process?

Start small, with one process that comes up every week. Choose how: from half an hour behind the scenes to a session at your offices.

01

Look behind the scenes

Free

Half an hour, by invitation

See everything we build running in our sandbox, the Proeftuin. No sales pitch, unless you ask for one.

Request an invitation

02Recommended

Introductory call

Free

Half an hour on Teams

You tell us what you already do with AI. We tell you what fits, or whether it is still too early.

Call with Marcel Logtenberg

Book a call

03

Get started

From €495

At your offices, excluding VAT

An AI quick scan, an AI assessment or an AI assessment plus inspiration session, at a fixed price.

Get started