We run our company as an org chart of AI agents. This week we read a paper on loop engineering. Most of it confirmed choices we had already made. One claim did not, and it was the load-bearing one: in an agentic system, generation is nearly free, and judgment is the scarce resource. An agent asked to grade its own work rubber-stamps it. The fix is not a better self-critique prompt. It is a separate, skeptical evaluator that does not read the output and nod, but acts: it runs the code, re-derives the numbers, re-opens the cited sources, and judges against a written bar. Verification is the floor of the loop, not a decoration on top.
That landed on a real gap. Our pipeline already reserved an evaluator slot: every department head could spawn a reviewer, and the delivery pipeline had a review stage. But the reviewer itself was never defined, so reviews quietly fell back to a generic one. We filled the slot. The new reviewer starts from REJECT, never praises, and verifies by acting, branching by work type (build and tests for code, re-derived numbers and re-read sources for research and compliance) before ending with an explicit PASS or REJECT. It runs on a different model tier than the agents that produce the work, so it does not inherit their blind spots. And every delegation now carries a checkable acceptance criterion, a done-when line, that the reviewer judges against.
Then we dogfooded it. The reviewer's first real job was our own data-protection compliance framework. It did not skim: it ran the encryption status check, verified our signed approval events, hit the engine's health endpoint, and read every file. It came back REJECT, catching three real consistency defects a self-grader would have waved through: a stale status in one document, an inconsistent name across notices, and a dead cross-reference. We fixed all three. (The framework is still pre-adoption, pending qualified legal review, and we hold no certification; the point here is the reviewer, not the framework.)
The lesson we keep relearning: correctness is not a step you bolt on at the end, it is the floor you build on. Generation is cheap now. Judgment is the part worth engineering. On its first run, the evaluator earned its keep.
I write about agentic systems, evaluation, and verifiable engineering as metaend. If you want to reach me, build on any of this, or tell me where it breaks, the door is here:
More writing lives at paragraph.com/@metaend.