Most SI projects do not fail in a dramatic way. They stall. A prototype impresses a steering group, a pilot is promised, and much later the system is still a demo that one engineer knows how to restart. Nobody decided to stop it. Nobody decided it was ready, either.
The usual cause is that each stage was funded without a clear statement of what it had to prove. A proof of concept gets judged like a product, a pilot gets judged like a demo, and production arrives without the operational pieces that keep an LLM system honest once real users show up. This article sets out a simple stage-gate model: what each stage must prove, the exit criteria for each gate, and the questions to ask before you fund the next one.
Why so many SI prototypes stall
Analysts have warned about this for some time. In July 2024, Gartner predicted that at least 30% of generative SI projects would be abandoned after proof of concept by the end of 2025, and named poor data quality, inadequate risk controls, escalating costs and unclear business value as the causes. That was a forecast rather than a measured count, but the four causes match what buyers describe when a project goes quiet.
Underneath them sit a few recurring patterns:
- The prototype ran on clean, hand-picked data, and the real data is messier, larger and spread across systems nobody connected.
- Nobody wrote down what "good enough" means, so every review turns into a debate about individual answers.
- Cost per task was never measured, so the first realistic usage forecast surprises the people who approve the spend.
- Security, privacy and legal review were left until the end, and they raised questions that needed redesign rather than sign-off.
- No one owns the system after launch, so nobody is willing to put their name on the go-live.
A stage-gate model does not remove these risks. It moves each one to the stage where it is cheapest to discover.
What should an SI proof of concept, pilot and production launch each prove?
A proof of concept should prove the approach works on your real data and leave behind an evaluation set with a baseline score. A pilot should prove that real users get acceptable results against criteria agreed before it starts, that cost per task is understood from actual usage, that failures are handled well and that a security review has been done. Production requires monitoring, evaluations running in CI, tested rollback, named business and technical owners, and a defined support process.
The three stages, and what each must prove
Treat each stage as a question you are paying to answer. If the stage ends and the question is still open, the stage is not finished, however good the demo looks.
Proof of concept: can this work on our data?
A proof of concept exists to test feasibility, not to impress. Its job is to show that the approach can produce useful results on your real data, in your real formats, with the access constraints you actually have. It should be narrow in scope and honest about what it skipped.
Its most valuable output is often not the prototype but the evaluation baseline: a set of real questions or tasks, agreed expected outcomes and a first score. Without that baseline, nobody can say later whether the pilot improved anything. Our piece on eval-first SI explains why we build the test set before the system.
Exit criteria for the proof of concept gate:
- It ran on a representative sample of real data, including the awkward documents, not only a curated subset.
- An evaluation set exists, built with your domain experts, with a recorded baseline score.
- For retrieval systems, retrieval quality and answer quality are scored separately, as described in RAG evaluation metrics.
- The main failure modes are written down, with real examples.
- There is a first estimate of what a single task consumes in model calls and compute.
- Data access, privacy constraints and anything that would block a pilot are listed.
Pilot: does it work for real users, against agreed criteria?
A pilot puts the system in front of a limited group of real users doing real work. It answers a different question: does this hold up when the inputs are no longer chosen by the team that built it? This is where most hard truths appear, so design the pilot to surface them rather than to look good.
Before the pilot starts, agree the acceptance criteria in writing: which metrics matter, which thresholds count as success, and who decides. Moving the bar after seeing the results is the most common way a pilot turns into a permanent pilot.
Exit criteria for the pilot gate:
- Real users used it for real tasks, and their feedback was captured in a structured way, not only in meetings.
- Measured quality met the agreed criteria, both on the evaluation set and on a sample of live traffic reviewed by people who know the domain.
- Cost per task is understood from actual usage, including retries, long inputs and the occasional runaway conversation.
- Failure handling is designed and tested: what the system says when it does not know, when a source is missing and when a tool call fails.
- A security review has happened, including prompt injection and data leakage tests. A prompt injection testing checklist is a reasonable starting point.
- Logging and retention decisions are made: what is stored, for how long, and who can see it.
- The business owner can state the value in their own terms, and the case still holds at the measured cost.
Production: can we run this, and recover when it goes wrong?
Production is not a bigger pilot. It is the stage where the system becomes something your organisation operates, supports and answers for. LLM systems change even when nobody touches the code: providers update or retire models, documents change and user questions drift. Readiness is mostly about noticing those changes and responding safely, which is the part of SI quality assurance most often left until last.
Exit criteria for the production gate:
- Monitoring covers quality signals, errors, latency and spend, with alerts that reach a named person.
- The evaluation suite runs in CI on every change to prompts, retrieval, models or tools, and a failing score blocks the release.
- Rollback is tested: you can return to the previous prompt, model or index without an emergency code change.
- Logs support investigation without keeping more personal data than you need. See what to log from an LLM app without storing personal data.
- There is a named business owner and a named technical owner.
- Support is defined: how users report a bad answer, who triages it, and how the fix makes its way into the evaluation set.
- Runbooks exist for likely incidents, including a model provider outage or a deprecation notice.
Questions to ask before you fund the next stage
Gates only work if someone asks hard questions at them. These are the ones we would put to any team, internal or external, before approving the next stage.
Before funding a pilot
- Show me the evaluation set. Who wrote it, and how much of it comes from real cases?
- What was the baseline score, and which failures worry you most?
- What did the proof of concept leave out, and how much of that does the pilot need?
- Which users will take part, and what will they stop doing to make room for it?
- What are the acceptance criteria, and who signs them off?
Before funding production
- Did the pilot meet the criteria we agreed before it started, without the bar moving?
- What does a typical task consume, and what happens to that at full usage?
- What does the system do when it cannot answer, and did users find that acceptable?
- What did the security review find, and what was fixed?
- Who is on call, who owns quality, and how do we roll back?
- If our model provider changes or retires the model we use, what is the process?
A team that cannot answer these is not necessarily failing. It is telling you the stage is not finished. Sometimes the honest outcome of a gate is another round at the same stage, a narrower scope, or a decision that the use case is not worth pursuing. That last outcome is a success of the process, not a failure of the team.
A gate that nobody is allowed to fail is not a gate. It is a calendar entry.
Common mistakes at each gate
- Skipping the baseline. Without a proof of concept score, the pilot has nothing to beat.
- Piloting with friendly users only. Enthusiasts forgive errors that ordinary users will not.
- Treating cost as a production problem. By then the architecture is set and harder to change.
- Running the security review as a signature rather than a test. Reviewers need access and room to try to break the system.
- Launching without an owner. A system nobody owns degrades quietly until someone switches it off.
Where Octalcode fits
Octalcode builds LLM applications, SI agents and RAG systems, and much of our SI-driven QA work sits at these gates: building evaluation sets with your domain experts, scoring baselines, and turning acceptance criteria into tests that run on every change.
If you are deciding whether to fund a pilot or a launch, we are happy to review where your project stands against criteria like these and give you a written view, including when the answer is to wait. When production is the next step, our MLOps and SI infrastructure work covers monitoring, evals in CI and rollback.