What a good AI critic actually catches.
When I added a reviewer agent, the interesting part wasn't whether it found problems. It was which problems, and why a reviewer that sees too much stops seeing anything.
When we started adding reviewer agents to check other agents' work, I expected them to catch typos and broken layouts. They did some of that. But the most valuable things they caught were different, and more worrying: things that looked great and weren't true.
What it caught
In a run producing marketing material for a test company, the reviewer rejected drafts for three kinds of problem:
- Invented proof. Product claims and evidence that sounded convincing and simply didn't exist.
- Overclaimed certainty. Copy describing the review process as more independent and thorough than it actually was.
- Unsupported voice. Statements written as if from the founder, or a timeline of events, that nobody had provided.
Why these are the dangerous ones
Notice that none of these look wrong. They're fluent, on brand and persuasive. A quick skim would wave them through. That's exactly what makes them dangerous: a typo embarrasses you, but an invented claim about your product can get you in real trouble with a customer.
It's also what AI is naturally inclined to produce. Models are trained to write what good marketing copy looks like, and good marketing copy is full of confident proof. If the proof isn't there, the model writes it anyway.
Why the reviewer has to meet the work fresh
The same run taught a second lesson. A reviewer that reads the producer's reasoning before looking at the work tends to agree with it. The explanation is persuasive, so the reviewer starts inside the producer's way of seeing things and checks the details while never questioning the frame.
The reviewers that caught real problems started from the goal, the company's approved facts and the finished work itself. The producer's reasoning was there if they needed it, but it wasn't the first thing they read.
And it has to be able to say no
A review that can't send work back is just commentary. The reviewer's rejection has to actually stop the work being accepted, with the specific evidence of what's wrong, so the next draft fixes the real problem.
It shouldn't fix the work itself, though. A reviewer that quietly rewrites the draft and then approves its own version has stopped being a reviewer.
The honest caveat
Reviewing isn't free. Adding a critic made that run cost several times more and take much longer, and several other things changed during the run, so I can't claim the critic alone made the result better. What I can say is that the problems it caught were real, and they were the kind I'd least want a customer to find.
So I don't put a reviewer on everything. I put one where a confident, invented claim would be expensive: anything customer-facing, anything with facts about the product, anything that goes out under someone's name.