AI agent testing
AI agent testing is the discipline of verifying that an AI agent behaves correctly across the range of inputs and situations it will encounter in production. Because agents combine language models, tools, memory, and multi-turn dialogue, testing them requires techniques that go well beyond conventional software testing — the input space is too large to enumerate and the output space includes natural language that can be correct in many different phrasings.
The goal of AI agent testing is not to prove that the agent is perfect. It is to build confidence that the agent works well on realistic inputs, degrades gracefully on unfamiliar ones, and does not regress silently when the model, prompt, or tools change.
What makes agent testing hard
Three properties of AI agents complicate testing. First, non-determinism: the same input can produce different outputs across runs, especially at higher sampling temperatures. Deterministic assertions ("the agent returns X") give way to distributional ones ("the agent returns something semantically equivalent to X at least 95% of the time"). Second, multi-turn state: many failures only appear after several conversation turns, and testing conversations one turn at a time misses these. Third, tool integration: a good agent that talks to a broken API produces bad outcomes, so tests must cover the full stack, not just the LLM's outputs.
Categories of AI agent testing
Serious agent testing typically layers four categories of tests.
Unit tests exercise individual components — a single prompt template, a single tool, a routing rule — with deterministic inputs and outputs. They catch regressions in the deterministic parts of the system quickly and cheaply. They do not catch behavioral issues in LLM outputs, but they do verify the plumbing around the LLM is intact.
Golden-set evals run the agent against a curated dataset of representative examples, each with an expected outcome or quality rubric. Each example specifies inputs (conversation history, user request, available context) and a way to check the output — either a reference answer, a semantic-similarity threshold, or an LLM-as-a-judge scoring prompt. Golden sets grow over time as new production failures are added.
Simulations run the agent against a simulated user that generates realistic multi-turn dialogue. This uncovers issues that only appear across turns: memory bugs, escalation misfires, tool-call loops. Simulation testing is especially important for customer service agents where a single conversation may span 10 to 20 turns.
Regression suites capture historical conversations — often edge cases or resolved incidents — and re-run them after every change to make sure the agent still handles them correctly. This is the primary protection against silent regressions from prompt tweaks, model updates, or tool changes.
Simulation testing in depth
Simulation deserves special attention because it is uniquely powerful for agent testing. In a simulation test, one LLM plays a synthetic user (with a defined persona, goal, and personality) and another LLM plays the agent under test. The two exchange messages until the simulated user's goal is achieved, the conversation is escalated, or a turn limit is hit. A separate evaluator LLM or rubric then scores the conversation on dimensions like task completion, tone, accuracy, and policy adherence.
Simulations produce hundreds or thousands of realistic conversations for a fraction of the cost of hand-writing test cases. They surface emergent behaviors — how the agent handles a frustrated user, how it recovers from a wrong tool call, how it degrades when the knowledge base doesn't contain the answer — that no static test dataset would find.
Simulation testing is central to Decagon's own product surface: our AI agents are tested against thousands of simulated customer conversations before each release, and every production escalation flows back into the simulation library.
What to measure
Agent testing produces metrics across several axes. Task completion measures whether the agent achieved the user's goal; deflection rate and containment rate are common variants. Accuracy measures whether tool calls used correct arguments and whether the agent's factual claims match ground truth. Compliance measures whether the agent followed policy — respected escalation thresholds, avoided prohibited topics, disclosed AI status when required. Efficiency measures how many tool calls, LLM tokens, and turns were needed. User experience proxies — sentiment, sentiment change over the conversation, and CSAT — indicate how the interaction felt.
The right mix depends on the agent's job. A customer service agent's most important axis is usually task completion combined with compliance. A research agent's most important axis is factual accuracy. A coding agent's most important axis is whether the generated code runs and passes the reference tests.
Testing in the deployment lifecycle
Effective agent testing runs at several points in the deployment lifecycle. Pre-commit tests run on every code and prompt change, using fast unit tests and a subset of the eval set. Pre-release tests run the full eval set and simulation suite before promoting a new version. Shadow tests run the new agent version silently in parallel with the current production version, comparing their behaviors on real traffic without exposing users to the new version. Post-release monitoring tracks the same metrics on live traffic and alerts on regressions.
The best agent-testing organizations treat evaluations as first-class engineering artifacts: versioned, reviewed, expanded on every incident, and integrated into CI. See AI evaluation for the broader discipline.
