A simple way to validate what AI produces, and keeping the guardrails up as models change

Check the output the way you'd check a new employee's work – a sample, a written standard and a second pair of eyes on anything that matters. Guardrails keep up with model changes when they're tests you can run again, not instructions you hope get followed.

By Matthew Steele4 min read28 October 2026

Two colleagues at a desk comparing a printed draft against a short checklist, one ticking items with a pencil
Illustrative scene, not a customer's premises.

You can't read everything, so sample it

Nobody can read everything an agent produces, and trying to is usually how checking quietly stops. So decide the share: 100% for the first fortnight, a quarter for the next month, 1 in 10 after that, and back up again whenever something changes. Decide the standard – a short written list of what a good output has and what it must never have. And decide the reader, which should be the owner, the person who actually cares about the job. That's the whole system, and it's the same one you'd use for a new employee.

The three layers

Layer What it catches Cost Where to use it
Rules The checkable things: numbers in range, fields filled, forbidden words, lengths, dates in the future. Nearly free, runs on everything. Every agent.
Sampled reading Tone, sense, whether it answered the real question – the things a rule can't see. An hour of the owner's week. Every agent, with the share set by how far you trust it.
Second reader Mistakes that would cost you money or a customer. A person's time on each item. Anything that reaches a customer, moves money or changes a record that matters.

Most businesses have the middle layer, informally, and neither of the others. The rules layer is the one that pays for itself, because it never gets tired.

Guardrails that survive a model change

Build, test on the saved cases, publish, sample, and back to the casesA loop of four steps: build or change, run the saved test cases, publish, sample the output. Any change to the model, the instructions or the material sends it back to the test cases. The three layers of checking sit beneath: rules on everything, a sampled read, a second reader on anything that reaches a customer or moves money.Build or changethe agent, its instructions,or the material it readsRun the saved casesfive to twenty real exampleswith the right answer knownPublishversion up, change note,old version kept a weekSamplethe owner reads a share,falling as trust risesA model update, a changed prompt or a new document: back to the cases before anything goes live.Rulesnumbers in range, fields present, nothing from the never-list; runs on everythingA sampled readtone, sense, the real question; an owner's hour a weekA second readeranything reaching a customer or moving money
Build, test on the saved cases, publish, sample, and back to the cases whenever the model, the instructions or the material change.

An instruction like "never quote below the minimum margin" is really a hope. A model update, a changed prompt or a new document can make the tool ignore it, and you find out from a customer. A test is different – 5 saved cases where you know the right answer, run again every time anything changes, the model included. If the quoting agent still refuses the under-margin case after the update, the guardrail held. If it doesn't, the agent stays off until it does.

So the guardrail is the test, kept with the agent by its owner, the way one source of truth describes. Models will keep changing. The tests don't mind, you just run them again.

Closing the gap between what was wanted and what came out

In our experience the gap is nearly always this: the requirement lived in someone's head as "I'll know it when I see it", and the builder built something else. The habit that closes it is the acceptance test, written before anything gets built, by the person who wants the thing:

  1. Five real cases from last month, including two awkward ones.
  2. For each one, what a good output looks like. Not a description – the actual output, or a marked-up version of an existing one.
  3. A short list of what would make an output unacceptable.

The builder's job then becomes making those five cases pass. If the person asking can't write the 5, the requirement isn't ready yet, and that's worth knowing before 6 weeks of building. This one habit removes most of the "that's not what I meant" conversations, because the disagreement happens on day one over a sheet of paper instead of on day 40 over a tool.

Keeping it simple enough to actually happen

The simplest version that works is a shared sheet per agent – the five cases, the standard, the sample share, and a column for this week's check with the owner's initials. If it needs software to run, it'll stop running. A sheet and a diary entry survive.