How to Evaluate AI Agent Output Without Calling Another LLM

Here is the default approach to evaluating agent output in 2026: take the output, send it to another LLM, ask that LLM to judge quality, and trust the result. This is the approach most eval frameworks use. And it has two problems that nobody talks about enough. First, it is slow and expensive. Every evaluation requires an LLM inference call. That is $0.01 to $0.05 per eval, depending on the model and output length. If you are running an agent in production handling hundreds of requests per hour, you are paying for two LLM calls per request — one to do the work and one to check the work. Your eval costs start approaching your inference costs. Second, it is recursive. Who evaluates the evaluator? If GPT-4o judges your agent's output and says it looks good, what happens when GPT-4o is wrong? You could add a third LLM to check the second one, but that way lies madness and an exponential cloud bill. There is a better approach for a large class of eval checks. You do not need an LLM to tell