OpenAI

How to test code that calls the OpenAI API

Your code calls the OpenAI API — to summarize, classify, extract structured data, or drive an agent. How do you test it without flaky assertions, slow runs and a bill on every push? This guide compares the options and walks through a setup that works for Python and TypeScript.

Your options

Mocks are still the right tool for small units — "does my parser handle a refusal?" For anything that depends on what the model actually says, a recording of the real response is more honest than a mock you wrote yourself.

The key idea: point the SDK somewhere else

The official OpenAI SDKs read the OPENAI_BASE_URL environment variable. If it's set, every request goes there instead of https://api.openai.com/v1. That's all a record/replay proxy needs — no code changes, no SDK plugin.

retake sets that variable for the command it runs:

OPENAI_API_KEY=sk-... retake record -- pytest   # once: real calls, saved to cassettes/
retake replay -- pytest                         # every time after: offline and free

Python and pytest

A function under test:

from openai import OpenAI

def classify(client: OpenAI, ticket: str) -> str:
    response = client.chat.completions.create(
        model="gpt-4.1-mini",
        messages=[
            {"role": "system", "content": "Classify the ticket as billing, bug or other. Reply with one word."},
            {"role": "user", "content": ticket},
        ],
    )
    return response.choices[0].message.content.strip().lower()

A fixture that gives each test its own cassette file, and a test:

import re, pytest
from openai import OpenAI

@pytest.fixture
def client(request):
    session = re.sub(r"[^A-Za-z0-9._/-]", "_", request.node.nodeid.replace("::", "/"))
    return OpenAI(default_headers={"x-retake-session": session})

def test_billing_ticket(client):
    assert classify(client, "I was charged twice this month") == "billing"

Record once with retake record -- pytest, commit cassettes/, and run retake replay -- pytest from then on.

TypeScript and Vitest

import OpenAI from "openai";
import { beforeEach, expect, test } from "vitest";

let client: OpenAI;
beforeEach(() => {
  const name = expect.getState().currentTestName ?? "default";
  client = new OpenAI({ defaultHeaders: { "x-retake-session": name.replace(/[^A-Za-z0-9._/-]+/g, "_") } });
});

test("extracts the invoice total", async () => {
  const total = await extractTotal(client, invoiceText);
  expect(total).toBe(1249.5);
});

retake record -- npx vitest run
retake replay -- npx vitest run

Responses API, streaming, structured outputs, embeddings

retake proxies HTTP, so it doesn't need to understand each endpoint — anything under the base URL is recorded:

• Responses API (/v1/responses) and Chat Completions (/v1/chat/completions) both work, including tool and function calls.
• Streaming (stream=True): server-sent events are stored as whole events and replayed byte-for-byte, so your streaming code sees exactly what it saw live.
• Structured outputs (response_format with a JSON schema): the schema is part of the request, so changing it is detected as a change, and the recorded JSON is what your parser is tested against.
• Embeddings (/v1/embeddings): replayed exactly, which makes retrieval in RAG tests deterministic too.
• Audio and files: binary bodies are stored as base64.

Tips

• Pin model versions. Use dated snapshot model names where available, so model changes happen when you decide.
• Ignore volatile fields. If you send a user or metadata value that changes every run, add --ignore user or --ignore metadata.request_id.
• Don't assert exact prose. Assert on structure and key facts, so re-recording doesn't break tests.
• Errors aren't recorded by default. 429 and 5xx responses are passed through but not saved, so SDK retries don't end up in your cassettes. To test your error handling with real error bodies, use --record-errors.
• OpenAI-compatible providers — Ollama, vLLM, Groq, OpenRouter and others — work the same way with --upstream name=url.