Guides
Testing LLM apps and AI agents
Practical, tool-agnostic guides for making tests of AI features fast, cheap and trustworthy. Each one ends with how to do it with retake, but the ideas apply whatever you use.
How to test AI agents deterministically
Why agent tests are flaky, how record/replay makes a whole multi-step, tool-calling trajectory repeatable, and what to assert.
Read the guide → CI/CDRun LLM tests in CI without API keys
Deterministic pipelines that work for fork pull requests, cost nothing per run, and still catch model drift with a scheduled live job.
Read the guide → OpenAIHow to test code that calls the OpenAI API
Chat Completions, the Responses API, streaming, structured outputs and embeddings — tested without paying for every run.
Read the guide → AnthropicHow to test Claude API code
Deterministic tests for the Anthropic Messages API: streaming events, tool use loops and extended thinking.
Read the guide → ConceptsVCR for LLMs: record/replay vs mocks vs live calls
The trade-offs of each testing strategy, why generic HTTP recorders struggle with LLM APIs, and how to choose.
Read the guide →