A simple way to validate what AI produces, and keeping the guardrails up as models change
Check the output the way you'd check a new employee's work – a sample, a written standard and a second pair of eyes on anything that matters. Guardrails keep up with model changes when they're tests you can run again, not instructions you hope get followed.

You can't read everything, so sample it
Nobody can read everything an agent produces, and trying to is usually how checking quietly stops. So decide the share: 100% for the first fortnight, a quarter for the next month, 1 in 10 after that, and back up again whenever something changes. Decide the standard – a short written list of what a good output has and what it must never have. And decide the reader, which should be the owner, the person who actually cares about the job. That's the whole system, and it's the same one you'd use for a new employee.
The three layers
| Layer | What it catches | Cost | Where to use it |
|---|---|---|---|
| Rules | The checkable things: numbers in range, fields filled, forbidden words, lengths, dates in the future. | Nearly free, runs on everything. | Every agent. |
| Sampled reading | Tone, sense, whether it answered the real question – the things a rule can't see. | An hour of the owner's week. | Every agent, with the share set by how far you trust it. |
| Second reader | Mistakes that would cost you money or a customer. | A person's time on each item. | Anything that reaches a customer, moves money or changes a record that matters. |
Most businesses have the middle layer, informally, and neither of the others. The rules layer is the one that pays for itself, because it never gets tired.
Guardrails that survive a model change
An instruction like "never quote below the minimum margin" is really a hope. A model update, a changed prompt or a new document can make the tool ignore it, and you find out from a customer. A test is different – 5 saved cases where you know the right answer, run again every time anything changes, the model included. If the quoting agent still refuses the under-margin case after the update, the guardrail held. If it doesn't, the agent stays off until it does.
So the guardrail is the test, kept with the agent by its owner, the way one source of truth describes. Models will keep changing. The tests don't mind, you just run them again.
Closing the gap between what was wanted and what came out
In our experience the gap is nearly always this: the requirement lived in someone's head as "I'll know it when I see it", and the builder built something else. The habit that closes it is the acceptance test, written before anything gets built, by the person who wants the thing:
- Five real cases from last month, including two awkward ones.
- For each one, what a good output looks like. Not a description – the actual output, or a marked-up version of an existing one.
- A short list of what would make an output unacceptable.
The builder's job then becomes making those five cases pass. If the person asking can't write the 5, the requirement isn't ready yet, and that's worth knowing before 6 weeks of building. This one habit removes most of the "that's not what I meant" conversations, because the disagreement happens on day one over a sheet of paper instead of on day 40 over a tool.
Keeping it simple enough to actually happen
The simplest version that works is a shared sheet per agent – the five cases, the standard, the sample share, and a column for this week's check with the owner's initials. If it needs software to run, it'll stop running. A sheet and a diary entry survive.