Most eval harnesses we inherit on day one of an engagement are built to answer "does this response sound plausible" rather than "did the model do the thing correctly." Those are different questions, and only one of them tells you whether it's safe to change the prompt, swap the model, or ship the next version.
Before we trust an eval harness enough to gate a deploy on it, we run it through the same four checks every time.
1. Does it fail on cases you know are wrong?
The single most common problem: an eval set with no negative examples. If every case in the suite is a real, correctly-handled request, a harness that always returns a fixed canned answer can still score close to 100%. We seed at least a third of the set with inputs we know should fail — ambiguous requests, adversarial phrasing, cases the current model already gets wrong — specifically to confirm the scoring function can tell the difference.
def score(response: str, expected: Expectation) -> float:
if expected.must_contain and expected.must_contain not in response:
return 0.0
if expected.must_refuse and not looks_like_refusal(response):
return 0.0
return semantic_similarity(response, expected.reference)A surprising amount of "eval harness" code we see is just the last line of that function — pure semantic similarity against a reference answer, with no hard failure conditions above it. That catches drift in tone. It does not catch a model confidently returning the wrong account balance.
2. Is the eval set actually yours?
Generic benchmarks tell you how a model performs on the internet's idea of your problem. They don't tell you how it performs on your actual traffic, which has its own weird distribution of edge cases: the customer who pastes in a support ticket instead of a question, the request in a language your prompt wasn't tuned for, the input that's technically valid JSON but structurally nonsense. We build the eval set from real logged requests — sanitized, but real — every time.
3. Does a regression in the eval mean a regression for users?
We run this one backwards: take a change everyone agrees made things worse — a prompt edit that was rolled back, a model downgrade that got reverted — and confirm the eval score actually drops when we replay it. If the harness doesn't notice a change that the team already knows was bad, it isn't measuring the thing that matters, no matter how good the pass rate looks.
4. Who looks at the failures?
An eval harness that reports a number to a dashboard nobody opens isn't a safety mechanism, it's a decoration. Before we sign off on one, there's a named person whose job includes reading the failing cases after every run, not just the aggregate score. Usually that turns up a category of failure the eval set doesn't cover yet, which goes back into check 2.
None of this is exotic. It's the same instinct as a good test suite: assert on outcomes, not vibes, and make sure something fails when it's supposed to. The harness that survives all four checks is the one we're willing to gate a production deploy on. Most of what we're handed on day one survives one or two.