Most AI features don't fail because the model is bad. They fail because nobody decided what "good" looks like, so nobody notices when it quietly gets worse. A demo that impresses in a meeting can still embarrass a client in front of their customers a month later.
Before an LLM goes anywhere near production, we run every AI feature through three checks. They're simple, and they've kept more than one project from launching something nobody could trust.
Check one: is AI the right tool?
The first question is whether the problem needs a language model at all. A surprising number of "AI features" are really search, a better form, or a rule someone hasn't written down yet. Those are cheaper, faster and much easier to explain.
A quick test: if a careful new hire with the same documents could do the job in a minute, a model probably can too. If that new hire would need to call three people first, the problem is the process, not a missing AI feature.
We reach for an LLM when the input is messy human language and the output can be checked, either by a person or by code. Summarising support tickets, drafting replies for an agent to edit and pulling fields out of documents are good fits. Making final decisions about money or health is not, unless a person stays in the loop.
Check two: can we measure it?
If we can't tell whether an answer is good, we can't ship it. So before writing a single prompt we collect a small test set: fifty to a hundred real examples, each with what a good answer has to contain. The client's team writes or reviews most of them, because they know their customers better than we do.
Every change to a prompt or a model then runs against that set. We use prompt diff, a tool from our lab, to see exactly which answers changed:
# one test case: a real question and what a good answer needs- question: "Can I change my delivery address after paying?" must_mention: ["order settings", "before dispatch"] must_not_mention: ["refund"] max_words: 120An AI feature without a test set is a demo, not a product.
Check three: what happens when it's wrong?
It will be wrong sometimes. The design question is what that costs and who notices. For every feature we sketch the failure path as carefully as the happy path:
- People can always see where an answer came from
- Low-confidence answers are labelled, not hidden
- There's a one-click way to flag a bad answer, and someone reads the flags every week
- A person makes the final call on anything that can't be undone
These are interface decisions as much as engineering ones, which is why our designers sit in on every AI project from the first workshop.
What ships with every AI feature
By launch day a client gets more than a prompt. They get the test set, a view of flagged answers, an estimate of the monthly cost per thousand users, and a short runbook for switching providers if prices or quality shift. If we can't hand those over, the feature isn't finished.
We also agree, before launch, on the numbers that would make us switch the feature off: a flag rate, a cost ceiling, a drop in a quality score. Having that line written down turns a scary conversation into a routine one.
The short version
Use AI where language is messy and answers can be checked. Measure before you ship. Design the failure path on purpose. Trust isn't something you add at the end; it's what you get when all three checks happen early.