Write the evals first

Teams often build an agent the way they build a demo: write a prompt, try a few examples, and tweak until it looks right. That works until the first change breaks something nobody checked. The fix is old and unglamorous: tests. For agents, they're called evals.

Start with real cases

Collect 30 to 50 real examples of the work: past tickets, invoices, requests, whatever the agent will see. Real cases beat invented ones because they contain the mess your users actually produce. For each one, write down what a good outcome looks like.

Then sort them into groups: the common cases, the awkward ones, the ones where the right answer is 'hand this to a person', and a few adversarial ones, such as a request that tries to talk the agent into something it shouldn't do.

How big should the set be?

Start small. Thirty to fifty real cases are enough to catch the obvious problems and to compare two versions of the same agent. Grow the set from there: every production failure adds a case, and every new capability adds a group.

Keep a slice, perhaps a fifth, that you never look at while tuning. It's your honest check that improvements are real and not just fitted to the examples you've been staring at.

A worked example: refund triage

Here's how the groups might look for an agent that handles refund requests.

Check more than the final answer

An agent can reach the right answer the wrong way. Good evals look at three things:

• The outcome. Is the answer correct? Use exact checks where you can, such as the right category or the right amount, and a rubric-based grader where you can't.
• The path. Did it call the right tools, in a sensible order, and avoid the ones it shouldn't have used?
• The budget. Did it finish within the step, time, and cost limits you set?

id: refund-over-limit
input: "Order A-2291 arrived broken. I want a full refund."
context:
  order_total: 240.00
  refund_limit_without_approval: 200.00
expect:
  tools_called: [orders.get, helpdesk.draft_reply]
  tools_not_called: [billing.issue_refund]
  outcome: approval_requested
  reply_mentions: [refund_or_replacement]

Using a model as the grader

Some qualities, like tone or completeness, can't be checked with code. A model can grade them against a rubric, but treat it as an instrument that needs calibrating. Have a person label a sample, compare the model's grades to theirs, and tighten the rubric until they agree. Check again from time to time, because graders drift when the underlying model changes.

Make the evals a gate

Run the suite on every change to prompts, tools, or models, and block the release when something regresses. Report results by group, not as a single average: a high overall score can hide a group that fails every time.

Reading the results

A report by group, with pass rates across several runs, tells you where to work next. The numbers below are invented to show the format.

The overall average is 96%, and it hides the weak spot: unclear requests. That's where the next round of work goes. A single headline number would have told you everything was fine.

Where evals run

• On every change, before merge: a fast subset that catches obvious breakage.
• Before every release: the full suite, with several runs per case.
• On a schedule, against a sample of real production runs, to catch drift when models, data, or users change.

Keep learning from production

Once the agent is live, sample real runs, label the failures, and add them to the set. The suite should grow with every surprise. It becomes the most valuable asset in the project, because it lets you change models, prompts, and tools without fear.

Common mistakes

• Writing evals after the prompt is finished, so they only confirm what you already see.
• Tuning the prompt until the test set passes, then treating the score as proof. Keep some cases aside that you never tune against.
• Grading only the final text and ignoring which tools ran.
• Treating a single run as a result.

Not sure whether you need an agent at all? Start with Workflow or agent?.