AI agents

How to test AI agents deterministically

An AI agent is a loop: the model decides on an action, your code runs a tool, the result goes back to the model, and so on until it answers. That loop is what makes agents powerful — and what makes them so hard to test. This guide shows how to turn a real agent run into a fast, repeatable regression test.

Why agent tests are hard

A single LLM call is already non-deterministic. An agent multiplies the problem:

• Errors compound. If step 2 returns slightly different text, step 3 sees a different conversation, picks a different tool, and the run takes a different path entirely.
• Runs are long and expensive. A single agent test can make 5–50 model calls. Run that suite on every push and the bill and wall-clock time add up fast.
• Failures are hard to read. When a live agent test fails, was it your code, the prompt, a model update, or plain randomness? With live calls you often can't tell.
• Mocks don't scale. Hand-writing fake responses for a 12-step tool-calling conversation is tedious, and the fakes drift away from what the real model does.

Three ways to test an agent

These aren't exclusive. A healthy setup uses mocks for small units, record/replay for agent regression tests in every CI run, and a small scheduled live job (or an eval suite) to notice when the model itself changes.

Why record/replay works for agents

Recording works for agents because of a simple chain reaction:

1. On replay, the model's first response is exactly the recorded one.
2. Your agent code is deterministic given its inputs, so it calls the same tools with the same arguments.
3. If the tools return the same results, the next request to the model is byte-for-byte the request that was recorded — so it matches, and the second response is replayed too.
4. …and so on, to the end of the run.

The whole trajectory — every model call, tool call and tool result — replays exactly, in milliseconds, without a network connection. And the moment anything changes (a prompt, a tool's output, the order of steps), the next request no longer matches and you find out precisely where.

Step by step

This example uses retake, a record/replay proxy that works with any language. Install it:

curl -fsSL https://zecrari.github.io/retake/install.sh | sh

1. Give each test its own cassette

Send an x-retake-session header so every test records to its own file. Here's a TypeScript agent loop using the Anthropic SDK and Vitest:

import Anthropic from "@anthropic-ai/sdk";
import { beforeEach, expect, test } from "vitest";

let client: Anthropic;
beforeEach(() => {
  const name = expect.getState().currentTestName ?? "default";
  client = new Anthropic({
    defaultHeaders: { "x-retake-session": "agent/" + name.replace(/[^A-Za-z0-9._/-]+/g, "_") },
  });
});

test("refund agent checks the order before refunding", async () => {
  const result = await runRefundAgent(client, "Refund order 1042, it arrived broken");
  expect(result.toolCalls.map((c) => c.name)).toEqual(["lookup_order", "issue_refund"]);
  expect(result.toolCalls[1].input).toMatchObject({ order_id: "1042" });
});

2. Record once

ANTHROPIC_API_KEY=sk-ant-... retake record -- npx vitest run

retake sets ANTHROPIC_BASE_URL (and OPENAI_BASE_URL) for the test command, forwards every call to the real API, and saves the run to cassettes/agent/<test name>.json.

3. Replay forever

retake replay -- npx vitest run

No network, no key, no cost. If any call doesn't match a recording, the run fails with a diff.

4. Commit the cassettes

Request headers (where API keys live) are never stored, and key-like strings in bodies are redacted. Run retake check before committing to be sure.

Make your tools deterministic

The chain reaction only holds if your tools return the same results on every run. Common culprits and fixes:

• Clocks. A tool that returns "now" changes the conversation every run. Inject a fixed clock in tests.
• Databases and IDs. Seed test data, and avoid random UUIDs in anything the model sees.
• External APIs. Tools that call other services (weather, search, your backend) need their own fixtures — retake records LLM traffic, not every HTTP call your tools make.
• Request metadata. If your agent puts a request ID or timestamp in the model request itself, tell retake to ignore it: --ignore metadata.request_id.

What to assert

Because replays are exact, you could assert on the precise wording of the final answer. Don't — the next re-record will break it. Assert on behavior instead:

• The trajectory: which tools were called, in what order, with which key arguments.
• Structure: the final output parses, matches your schema, and contains the facts it must (an order number, a status).
• Side effects: what your code did with the answer — the refund issued, the ticket created.
• Guardrails: the agent did not call a dangerous tool, or stopped within a step budget.

When a replay diverges

When a request doesn't match, retake responds with a 404 (deliberately not a 5xx, which SDKs would silently retry) and prints a diff against the closest recording:

no recording in session `agent/refund` matches POST /anthropic/v1/messages (closest: step 3)
-          "content": "{\"status\":\"delivered\"}",
+          "content": "{\"status\":\"delivered\",\"eta\":null}",

Read the diff and decide:

• You changed something on purpose (a prompt, a tool's output shape): re-record with retake record, or retake auto to record only the new calls, and review the cassette diff in your pull request.
• You didn't mean to change anything: you just found a regression — before it reached production.

Catching model drift

Recordings freeze the model's behavior at the time you recorded. That's the point, but it means replay tests won't notice when the provider updates a model. Two habits cover the gap:

• Pin model versions in your code (dated snapshot names where your provider offers them), so changes happen when you choose.
• Run a scheduled live job — say weekly — that runs the suite with retake record against the real API. If it fails, or the re-recorded cassettes change meaningfully, you've caught drift. See the CI guide for a ready-made workflow.

Limits

• retake records HTTP calls to LLM APIs. It doesn't record tools that run in-process or over stdio (for example local MCP servers); give those deterministic fixtures.
• Record/replay tests your code's handling of real model output. It doesn't tell you whether that output is good — that's what evals are for.