Back

Tools & Frameworks

AI Evals

Effective AI evaluation starts with manually reviewing production traces — not with generic metrics dashboards. For agents specifically, the goal is usually floor-raising (eliminating failures that destroy trust) rather than benchmark-maxxing. Label ~100 traces by hand, categorize the failures, fix the obvious ones without an eval, then build targeted LLM-as-judge evals for the subjective problems that remain. Test the running agent path (tools, state, side effects), not isolated prompts. Keep eval suites small and grounded in real failures — prune cases that haven't failed in months. Containerized eval formats now let teams swap models, prompts, and agent versions against stable environments; coding agents can even automate the eval-authoring process itself.

Created May 28, 2026·Updated Sep 10, 2026

Recent Updates

Floor-Raising vs Benchmark-Maxxing

Success with AI products hinges on iterating fast across three activities: evaluating quality, debugging issues, and changing system behavior (prompts, fine-tuning, code). Most teams focus exclusively on changing behavior and plateau quickly — streamlining evaluation unlocks the other two.

Before designing evals, choose the frame. Benchmark-maxxing pushes top-end capability — useful when augmenting experts who can catch mistakes. Floor-raising eliminates the failures that destroy user trust — the right frame for agents that replace human workflows.

A finance agent that cross-references accounts and predicts spending is impressive. But if it sometimes can't answer "how much money is in my bank account?", users stop trusting it. Floor-raising addresses exactly this: making the agent reliable where reliability matters, in the workflows users actually run, in the moments where mistakes are expensive source(https://howtoeval.com).

Floor-raising is error analysis. You're not starting with an abstract test suite — you're doing detective work. Finding the places where the system bends, classifying them, and deciding which ones deserve engineering effort. A floor-raising eval suite is a memory of bugs you refuse to reintroduce.

The litmus test: if you could ship with 90% or 99% pass rate, the benchmark-maxxer chooses 99%. The floor-raiser asks "which 1% fails?" source(https://howtoeval.com).

One of the lowest-hanging floor-raising techniques: teaching your agent to refuse. If confidence is low, say "I don't know." Benchmark-maxxers hate this — it lowers their pass rate. Floor-raisers understand that a confident wrong answer is worse than an honest refusal.

Golden Cases

Before trusting an agent in production, write down 5–10 cases that represent your critical paths. Start with the simplest version: a single common user question the agent should always handle correctly. If your agent starts failing golden cases, you do not ship source(https://howtoeval.com).

For each golden case, don't just check the final output — inspect the full trajectory: user message, tool calls, retrieved context, reasoning chain. The path matters as much as the answer.

Evals as Product Management Artifacts

At Anthropic, the research product management team operates under the principle that "evals are the new PRDs." The traditional PM artifact — the product requirements document — assumed deterministic software where specifying inputs and expected outputs was sufficient. With non-deterministic AI systems, the eval is the specification: it captures user need, defines success criteria, and provides the measurement loop that researchers use to improve the model.

The process starts with user pain. Rather than running traditional user interviews, PMs read model interaction transcripts — full trajectories of what the user asked, what the model did, and where it failed. The guiding principle: sweat the tokens as much as you sweat the pixels. Failure modes in AI products have nuance that surface-level feedback obscures — a user reporting "Claude doesn't follow instructions" might mean hallucination, overconfidence, schema non-compliance, or something else entirely. Only reading the actual token-level trajectory reveals the real category.

Once a failure pattern is identified, PMs generate 30–40 examples of the failure and assemble them into an eval set — each with a prompt, a model response, and the expected correct answer. This eval set becomes the durable artifact: it enters the repository, runs against every new model version, and tracks whether the pain point has been resolved. Early Claude models, for instance, struggled with JSON schema compliance — a pain point that surfaced from user complaints about "not following instructions." The resulting eval now consistently passes at ~100%, confirming the issue is resolved.

This is essentially test-driven development for product management. The PM writes the test (the eval) before the fix ships, and the test remains as a regression guard. Because AI outputs are non-deterministic, evals must describe success broadly rather than matching exact outputs — making eval design a skill that blends product judgment with technical specification. The approach extends beyond model teams: any product built at the intersection of models, harnesses, context, and users benefits from evals as the core quality loop, since "you can't improve what you can't measure" and measurement in AI products requires staying close to the failure details. For more on how this fits into broader AI-native product development, see the team-level workflow patterns there.

Three Levels of Evaluation

Evals form a cost pyramid. Level 1 (unit tests) runs cheaply on every code change. Level 2 (human and model-based eval) runs on a set cadence. Level 3 (A/B testing) runs only after significant product changes. There's no strict formula for when to introduce each level — balance getting user feedback quickly, managing user perception, and product goals. But conquering Level 1 before moving to Level 2 is important, since model-based tests require more work and time to execute.

The Trace Review Process

The foundation of good evals is looking at production traces — the full interaction logs between users and your AI system, including system prompts, tool calls, and internal reasoning steps.

Start with 100 traces. Review them one by one, writing short notes about what went wrong (or right). The first few are slow; by trace 10 you're fast. The goal is theoretical saturation — keep going until you stop learning new things. Most teams find 100 is more than enough.

Don't root-cause yet. During review, just journal what you observe. "The conversation dead-ended." "The AI offered virtual tours but there are none." "Markdown formatting in a text message." Don't debug, don't propose solutions — just note the problems.

Use AI to categorize, not to observe. Export your notes into a spreadsheet or CSV, then ask an LLM to group them into 5–6 categories (axial codes). Refine the categories until they feel right. Then use the LLM to classify each note into a category. A pivot table instantly shows which problems dominate.

Asking Your Agent

One underused debugging technique: reconstruct the run exactly as it was passed to the agent, along with the latest response, and ask it directly what happened. This works because reasoning traces are often opaque — passing the trace back to the same model is the closest you can get to understanding its reasoning, since it can use the available trace, context, and prior messages as evidence source(https://howtoeval.com).

Ask something like: "You were wrong. The answer was X. What would I need to have changed for you to get this right?" The answer isn't always the truth, but it surfaces where the agent is misinterpreting or overindexing on specific parts of the prompt.

Two Types of Evals

Code-based evals (reference-based): When there's a clear right answer — dates, specific data lookups, format compliance — write deterministic assertions. These are cheap, fast, and don't need LLM judges. Fix the bug, write the test, move on. Many things are hard to produce but cheap to check once someone has done the preparation — the asymmetry of verification. If you have an expected output, every future evaluation run becomes a cheap deterministic check rather than an expensive LLM judgment source(https://langfuse.com/academy/evaluate/writing-evaluators).

Organize assertions by feature and scenario. For a real estate listing finder: "only one listing matches query" → assert len(results) == 1; "multiple matches" → assert len(results) > 1; "no matches" → assert len(results) == 0. Generic assertions (e.g., regex-checking that UUIDs aren't leaked in user-facing output) apply across all features source(https://hamel.dev/blog/posts/evals/). Hundreds of these tests, continuously updated as new failure modes appear in production data, provide rapid feedback when iterating on prompts or RAG.

Unlike traditional unit tests, a 100% pass rate isn't always required — the acceptable pass rate is a product decision depending on which failures you can tolerate.

LLM-as-judge evals (reference-free): For subjective quality — conversational flow, appropriate handoffs, tone — prompt an LLM to judge traces against specific criteria. These are more expensive but provide the most value for iterating on hard problems.

Not everything needs an eval. If the fix is obvious (e.g., the model doesn't know today's date because you forgot to include it), just fix it. Evals earn their keep on problems you'll iterate against repeatedly.

Code-Aware Offline Evals

Testing prompts in isolation makes no sense once the agent is entangled with code, tools, retrieval, permissions, and product state. The behavior lives in the whole system, not in the prompt string. Offline evals should look less like prompt scoring and more like ordinary software testing — Vitest, pytest, or jest source(https://howtoeval.com).

A good offline eval takes an input, runs the real agent path, and asserts on the result: output, tool calls, files changed, structured data, or final state. Sentry's vitest-evals demonstrates this with describeEval(...), an app-local harness, an explicit run(...), normal expect(...) assertions, and tool-call checks. OpenAI calls the same idea "macro evals" in their agentic systems cookbook: drive the real agent loop on representative inputs and grade the full trajectory source(https://howtoeval.com).

The key principle: evaluate the agent, not an LLM call.

Containerized Evals (Harbor)

The Harbor framework formalizes containerized evals as three components: an instruction (the task prompt given to the agent), an environment (a Dockerfile that installs tools, populates data, and sets up the filesystem), and a verifier that scores whether the agent completed the task correctly. Harbor runs the agent inside the environment and records its trajectory, artifacts, reward, and errors source(https://x.com/vtrivedy10/status/2079976006644072796/).

Containerized environments make the eval loop faster. The task and environment remain stable while the agent configuration changes — builders can swap models, tools, prompts, or complete agent versions and compare results directly. Multiple configurations can run in parallel. Reproducible environments are critical to signal quality: when an eval mirrors the relevant tools, data, permissions, state, and failure modes from production, builders get a stable testbed that is still representative of how the agent operates.

Reward Hacking

When iterating on verifiers, watch for reward hacking — agents taking shortcuts that satisfy the verifier without completing the task. Common patterns: overciting irrelevant sources to receive full credit, claiming an action never taken, exploiting exposed answer material, or satisfying a proxy metric without actually solving the problem source(https://x.com/vtrivedy10/status/2079976006644072796/).

The fix is to inspect both sides of the result: the agent trajectory (messages, tool calls, actions) and the verifier trajectory (evidence, reasoning, final score). This dual inspection reveals whether the task or verifier design is actually measuring what you care about. The first verifier is rarely the final one — the task, environment, and verifier typically go through several revision cycles.

Automated Eval Engineering

Coding agents can automate the eval-authoring process itself. LangChain's Eval Engineering Skill demonstrates this: the skill inspects how an agent is structured (prompts, models, tools, hooks), mines patterns from production traces (via tools like langsmith-cli), and proposes abilities to test. Crucially, the skill interviews the user rather than generating evals in one shot — users choose from proposed eval directions and specify which tools should run live versus be simulated (e.g., tool calls that incur costs or write to production) source(https://x.com/vtrivedy10/status/2079976006644072796/).

The resulting loop is: mine traces → identify a failure → build an eval → improve the agent → rerun. This aligns with the view that continual learning is a continuous data-mining problem where production data feeds evals that improve agents over time.

Building an LLM Judge

Write a judge prompt that specifies the exact criteria for pass/fail. For a human-handoff judge: list the specific failure modes (user requested transfer but was ignored, too many loops before handoff, etc.) and the conditions under which there is no failure.

Label real cases before writing the prompt. Evaluation criteria should not come exclusively from your head. Criteria drift is the catch-22: you need criteria to grade outputs, but grading outputs is what teaches you your criteria source(https://langfuse.com/academy/evaluate/writing-evaluators). Take 10–20 real cases of the failure mode you want to evaluate and label each one with a short comment. That's enough to build a robust judge. If you already did error analysis, most of this exists already.

Write the prompt like onboarding material. The bar: a new colleague could read it and reach the same verdicts you would source(https://hamel.dev/blog/posts/llm-judge/). A judge prompt has five parts source(https://langfuse.com/academy/evaluate/writing-evaluators):

  1. Context — what the application does and the domain knowledge needed to check the criterion.
  2. A precise criterion, including what to ignore — "Is the response high quality?" is a question two people would answer differently. "The response cites at least one source document. Ignore formatting issues." gets you the same answer every time.
  3. Labeled examples with reasons (optional) — 2–4 of your labeled cases mixing pass and fail. Start without them; add only when the judge isn't accurate enough, since they increase token consumption.
  4. Reasoning first, verdict last — reasoning-first prompting measurably improves judge accuracy source(https://eugeneyan.com/writing/llm-evaluators/). The reasoning is also the first thing you read when you disagree with a verdict.
  5. An explicit way out — let the judge answer "unknown" when information is missing instead of guessing.

Always output binary or categorical. Continuous scales (1–5) introduce enormous complexity. LLMs are poor at consistent numeric scoring — they even develop favorite numbers (GPT-3.5 has a documented preference for the number 7) — the results aren't actionable ("3.2 vs 3.7 — is that better?"), and they create false precision that erodes stakeholder trust. A pass/fail verdict is easily verifiable: you can count exactly how often the evaluator catches a failure and how often it clears a pass. There is no equivalent test for whether a 7 was the right score. When one event has several mutually exclusive outcomes, use a single categorical evaluator that picks one label per case (resolved / abandoned / handed_off) rather than several overlapping binary ones source(https://langfuse.com/academy/evaluate/writing-evaluators). If you can't define a clear pass/fail threshold, you probably don't understand the problem well enough yet.

Add explanations alongside the score. Have the judge output a structured response with both an explanation field and a binary score. The explanation helps you debug the judge's reasoning when it disagrees with your labels.

Measuring Judge Quality

The critical step most teams skip: validating that the judge actually works. You already have human labels from the trace review — use them as ground truth.

Never use raw agreement. If a failure occurs 10% of the time, a judge that always says "no failure" achieves 90% agreement. That looks great on a dashboard and is completely useless.

Use true positive rate and true negative rate instead. TPR: how often does the judge correctly identify real failures? TNR: how often does it correctly identify non-failures? A confusion matrix makes this concrete — you can see exactly where the judge is wrong and in which direction. (Note: raw agreement can be appropriate when failure classes are roughly balanced, around 50/50 — but this is rare in production systems.)

One practical alignment workflow: send a domain expert a spreadsheet with the model's response, the critique model's written critique, and the critique model's binary label. The expert fills in their own critique, label, and preferred response for 25–50 examples at a time. Iterate on the critique prompt until alignment stabilizes, then continue periodic spot-checks to monitor drift source(https://hamel.dev/blog/posts/evals/).

Whether the error rates are acceptable is a business decision, not a statistical one. False positives (flagging non-failures) are usually cheaper than false negatives (missing real failures). Look at the specific misclassified traces, iterate on the judge prompt, and re-measure.

Hold out data when adding examples. If you put few-shot examples in the judge prompt, don't test against those same examples — you'll get artificially perfect scores.

Runtime Judges

Most LLM-as-judge usage today sits outside the application execution path — offline evaluation against datasets. An emerging architectural shift moves judges inside the agent loop, where judgment informs control flow: continue, retry, route to another model, gather more evidence, or escalate to a human source(https://x.com/joshuacrosen/status/1929256835976724863). The judge is no longer just telling you whether the application worked yesterday — it's helping determine what the application does next.

This is a much bigger role. Mistakes from an offline judge are annoying; mistakes from a runtime judge become application failures — creating loops, blocking good work, approving bad work, and adding latency to every execution.

Specialized Judge Models

The runtime judge doesn't need to be your strongest frontier model. Judging a narrow property (e.g., "are all claims supported by the supplied evidence?") can be far easier than producing the original work. Specialized judge models like Prometheus and distilled evaluators trained for particular judgment types can make runtime judging economically viable at scale. Galileo has built evaluation models specifically designed to be cheap enough for continuous production-scale use source(https://x.com/joshuacrosen/status/1929256835976724863).

Decomposed Judgment

Rather than asking one model to make a single fuzzy "was this good?" decision, break the judgment into smaller decisions with well-scoped prompts and put structure around how they combine. G-Eval pioneered this by having the model generate evaluation steps from the criteria before scoring. DeepEval extends it with DAG-based evaluation — individual LLM judgments sit inside a larger deterministic decision graph source(https://x.com/joshuacrosen/status/1929256835976724863). This aligns with the anti-pattern advice against "God Evaluators": narrow judges checking one criterion each are easier to build, calibrate, and debug.

Pairwise Comparison

Models are better at deciding which of two outputs is better than assigning absolute scores. Pairwise judging takes advantage of this — given two outputs from the same input, the judge picks which better satisfies the criteria. LangSmith and DeepEval support this directly, including answer-order randomization to reduce position bias source(https://x.com/joshuacrosen/status/1929256835976724863).

At runtime, this enables architectures where an agent generates multiple plans and a judge selects the best, or two agents independently analyze a problem and a comparator picks the stronger result. A proposed action can be compared against an alternative before the system commits.

Multi-Judge Panels

A single judge is still an LLM — it can make mistakes for the same reasons the worker model can. Multi-judge architectures use panels, voting, and deliberation to increase confidence. MAJ-EVAL creates multiple evaluator agents representing different evaluation dimensions and lets them deliberate source(https://x.com/joshuacrosen/status/1929256835976724863).

The practical value for runtime architectures isn't necessarily the vote outcome — it's the disagreement signal. Agreement between independent judgments increases confidence; disagreement triggers retry with a stronger model, additional evidence gathering, or human escalation. This maps naturally to the retry and routing logic that runtime agents already need.

Meta-Evaluation

Once a judge affects application behavior, its reliability must be measured. LLM judges exhibit known biases: position bias (changing decisions based on answer order), style preference, difficulty distinguishing close-quality outputs, and self-preference for outputs resembling their own. Benchmarks for measuring these problems include LLMBar (instruction-following under adversarial conditions), JudgeBench (difficult response pairs with objective preferences), and RewardBench (reward models across challenging preference tasks) source(https://x.com/joshuacrosen/status/1929256835976724863).

Anthropic's Bloom project provides an accessible pattern: evaluate candidate judge models against human-labeled transcripts before choosing the production judge, then add a meta-judge that reviews evaluation results at a higher level. The simpler version any team can adopt today: periodically compare the judge against humans on a sample of production decisions and adjust the rubric, model, or decision boundary when disagreement is unacceptable.

Hybrid Deterministic + LLM Checks

The biggest advantage at the application layer is controlling the surrounding system. If something can be checked deterministically — schema validation, type checks, database lookups — check it deterministically. OpenAI's grader architecture formalizes this separation with model-based graders alongside deterministic graders (string checks, Python code), combinable into larger evaluations source(https://x.com/joshuacrosen/status/1929256835976724863).

Inside runtime agents, an LLM might judge whether evidence is sufficient or a recommendation well-supported. Deterministic code decides what combination of those judgments and other facts is required before the workflow proceeds. Drawing this boundary well — knowing where LLM judgment adds value and where determinism suffices — is one of the highest-leverage architectural decisions for agent builders.

Removing Friction from Trace Review

The biggest bottleneck in eval quality is friction in the data-review loop. Render traces in domain-specific ways that put all relevant context on one screen — the trace log, the CRM state, tool outputs, and links to upstream systems. Off-the-shelf logging tools (LangSmith, etc.) are a starting point, but custom viewing and labeling tools built with lightweight frameworks (Gradio, Streamlit, Shiny) in less than a day often outperform them for domain-specific needs source(https://hamel.dev/blog/posts/evals/).

Design choices that compound: make the final LLM output editable by a human so reviewed traces double as curated fine-tuning data. Add filters by feature and scenario. Embed the downstream application view so reviewers see exactly what the user saw.

Start with binary labels (good/bad) rather than granular scores — they're simpler to manage and less onerous for labelers. More advanced techniques (active learning, consensus voting) can come later.

Production Monitoring

Once a judge is validated, deploy it against production traces:

  • CI integration: Run judges on test cases whenever you change code or prompts.
  • Sampling in production: Score a random sample of live traces to monitor whether failure rates are trending up or down.
  • Debugging at scale: Use judges to find all instances of a specific failure type across production data, then drill into those traces.

Keep the eval suite small — typically under a dozen judges. Each one has maintenance cost (periodic re-labeling, prompt updates as the product evolves). Code-based evals are cheaper to maintain than LLM judges.

Scaling Review with Volume

The workflow should scale with traffic. At ten agent runs a day, read everything. At ten thousand, the system needs to tell you what deserves attention. The mistake is reaching for high-volume machinery before you have enough raw texture to know what you're looking for source(https://howtoeval.com).

Start with raw logs as the firehose — look for confusion, frustration, near-misses, repeated prompts, and "you're holding it wrong" moments. When the same stumble keeps appearing, promote it to an issue. Track long-horizon behavioral signals (refusal quality, ignored tool errors, context loss, user frustration). Once you understand an issue, ship the fix and compare affected metrics. Production tells you whether the change actually helped.

Eval Suite Hygiene

Not every bug deserves an eval case. Ask: is this a critical path? Could it regress? Is it representative of a class of failures, or truly one-off? Be ruthless about pruning — 20 high-signal cases beats 200 low-signal ones source(https://howtoeval.com).

A good heuristic: if an eval case hasn't failed in 3 months, it's either not testing something important or the agent has genuinely improved. Either way, question whether it needs to be there. The pattern for non-trivial issues remains: find in production, reproduce locally, add to evals, fix, verify, ship.

Plan to spend 10–20% of agent development time on evaluation and monitoring — not just writing eval cases, but reading traces, tuning signals, and investigating issues source(https://howtoeval.com).

Synthetic Data for Pre-Launch

Before you have real users, generate synthetic test inputs by defining dimensions — the axes of variation that matter for your product. For a property management assistant: customer type (resident vs. manager), property class (luxury vs. standard), interaction type (maintenance vs. leasing). Generate the cross-product, then ask an LLM to create plausible queries for each combination. This explores the input space far more effectively than asking an LLM to "generate some test questions." For more on generating realistic diverse user cohorts — including evolutionary optimization of persona-generating code — see Synthetic Personas.

Synthetic generation also works for unit test cases. Prompt an LLM to produce N variations of realistic user inputs for a specific feature, then pair each with a verification query. For a CRM contact creator: generate "Create a contact for John Smith (johndoe@apple.com)" paired with "What's the email address of John Smith?" — then assert the round-trip succeeds. One signal that your tests are good: when the model struggles to pass them, those failure modes become tractable targets for fine-tuning source(https://hamel.dev/blog/posts/evals/).

Eval Infrastructure Reuse

A well-built eval system generates two additional superpowers nearly for free:

Fine-tuning data. 99% of the labor in fine-tuning is assembling high-quality data with good coverage. If you have eval infrastructure, you already have synthetic data generators (reuse your test-case prompts), quality filters (Level 1 assertions and Level 2 critique models reject bad examples), and human-curated traces from the labeling tool. Fine-tuning is best for learning syntax, style, and rules; RAG is better for supplying context and up-to-date facts.

Debugging. When you get a complaint or observe an error, a robust eval system gives you a searchable trace database, assertion mechanisms that flag known bad behaviors, log navigation tools for root-cause analysis (RAG failure? code bug? model limitation?), and the ability to make a fix and quickly test its efficacy.

The Collapse of Harnesses

The current model of "agent SDKs that call models" is showing cracks. As models become more capable and agentic, the distinction between "the model" and "the agent" blurs — visible in Claude Code, the Cursor CLI, and emerging agent SDKs. When the agent is just a prompt and a model, with no framework code to instrument and no explicit tool loop to trace, the harness collapses into the model itself source(https://howtoeval.com).

This changes monitoring. Today we trace tool calls because we can. Tomorrow we might record input/output pairs and ask the model to explain what it did — like asking a human what they were thinking. Verification becomes harder.

The implication: end-to-end evaluation becomes even more important. If you cannot inspect the internals, you have to trust and verify the outputs. Golden cases and production monitoring become the only game in town.

Common Anti-Patterns

  • "God Evaluator": A single judge that rates accuracy, tone, and completeness together on a 1–10 scale source(https://langfuse.com/academy/evaluate/writing-evaluators). The resulting score doesn't tell you what to fix. Narrow judges that each check one criterion are also easier to build — getting a judge to agree with you on one specific criterion is a much lower bar than agreeing on "quality" source(https://eugeneyan.com/writing/product-evals/).
  • Generic metrics first: Starting with helpfulness/toxicity/coherence dashboards instead of looking at actual traces. These generic scores rarely match the real problems in your application.
  • Skipping annotation: Jumping straight to automated evals without manually reviewing traces. The annotation step is the single most valuable part of the process — even without building any judges, it produces actionable insights.
  • Continuous scoring: Using 1–5 scales instead of binary pass/fail. Adds exponential complexity for marginal benefit. Teams must be "extremely disciplined" to use Likert scales correctly, and most aren't.
  • Reporting raw agreement: Using agreement percentage instead of TPR/TNR. Misleading whenever failures are rare (which they usually are in a working system).
  • Over-investing pre-launch: Building elaborate eval suites before you have real users. Dog-food the product, get a few friendly customers, and save the serious eval infrastructure for when you have enough data that manual review can't keep up.
  • Focusing only on changing behavior: Jumping straight to prompt engineering or fine-tuning without investing in evaluation and debugging infrastructure. This is the most common root cause of AI products that never progress beyond a demo.
  • Massive eval suites: Adding every bug as an eval case without pruning. Six months later you have 500 cases, CI takes 20 minutes, and the team starts ignoring failures because "it's always something" source(https://howtoeval.com).

The Advanced Use for Generic Scores

Generic scores (hallucination, coherence, etc.) have one legitimate use: as a sampling mechanism. Sort your traces by a generic score, then manually review the highest-scoring ones to see if the score correlates with anything real. If it does, you've found a useful filter for prioritizing trace review. But never report generic scores directly to stakeholders.

Shadow Testing

Shadow testing (also called shadow runs) lets a candidate agent version process real production traffic while its output is shown to nobody. The candidate runs side-by-side with the live agent; only the live agent's responses reach users. This de-risks deployments by revealing failures under real-world input distributions before any user sees the new version. Shadow runs sit between offline evals and full A/B tests — they use real traffic (unlike offline) but carry zero user-facing risk (unlike A/B). Langfuse supports tracing shadow runs, comparing candidate outputs against the live baseline, and monitoring eval results across both.

Red Teaming

Red teaming attacks the system before an adversary does — probing for jailbreaks, prompt injection, data leaks, and tool abuse. Unlike other eval types that measure whether the agent does the right thing, red teaming measures whether the agent can be made to do the wrong thing. Garak scans LLM systems for vulnerabilities and unsafe behavior, automating adversarial probes that would otherwise require manual security expertise.

Eval Tooling Landscape

A non-exhaustive map of OSS tools to eval categories covered on this page:

  • Golden sets / regression suites: Promptfoo — repeatable eval suites with CI integration; catch regressions across prompt, model, or toolset changes
  • LLM-as-judge: OpenEvals — ready-made evaluator templates for LLM applications
  • Multi-dimensional scoring: DeepEval — custom metrics with independent scoring per dimension (correctness, tone, safety, cost)
  • Trajectory eval: AgentEvals — grades agent actions, decisions, and tool calls across the full run trajectory
  • Tool unit testing: MCP Inspector — inspect and test MCP server tools and responses in isolation, no model in the loop
  • A/B testing: GrowthBook — feature flags, controlled experiments, and product analytics for splitting real traffic between agent versions
  • Human review: Argilla — collect human feedback, review outputs, and build labeled datasets for judge calibration
  • Shadow testing: Langfuse — trace production and shadow runs, compare candidates, monitor eval results
  • Red teaming: Garak — automated adversarial scanning for LLM vulnerabilities

Sources

  • AI Evaluations Clearly Explained in 50 Minutes (Real Example) | Hamel Husain (Peter Yang, video) — foundational source; end-to-end walkthrough of the trace-review-to-LLM-judge pipeline using Nurture Boss (AI property management assistant) as a real-world case study
  • Your AI Product Needs Evals | Hamel Husain — three-level eval framework (unit tests → human/model eval → A/B testing); Rechat/Lucy case study; assertion-based unit tests; synthetic test generation; custom trace-viewing tools; eval infrastructure reuse for fine-tuning and debugging
  • How to evaluate AI agents | howtoeval.com — floor-raising vs benchmark-maxxing frame; golden cases; code-aware offline evals (test the agent, not the LLM); asking your agent directly; eval suite pruning; production monitoring at scale; collapse of harnesses
  • Towards Automating Eval Engineering | Viv (LangChain) — Eval Engineering Skill for coding agents; Harbor containerized eval format (instruction + Dockerfile + verifier); interview-driven eval design; reward hacking patterns; continual-learning-as-data-mining loop
  • 10 agent evals every AI engineer should know | elune — beginner-level overview of 10 eval categories with OSS tool recommendations; shadow testing and red teaming concepts; specific tooling landscape
  • Good evals are boring | Lotte (Langfuse Academy) — practical eval methodology: asymmetry of verification; "God Evaluator" anti-pattern; criteria drift; five-part judge prompt structure; categorical scoring; labeled-case calibration workflow
  • Evals are the new PRDs: how AI is rewriting the PM job | Dianne Penn (Lenny's Podcast, video) — Anthropic's PM workflow where evals replace PRDs; user-feedback-to-eval pipeline; "sweat the tokens" principle; JSON schema compliance as origin case study; test-driven development analogy for PM work
  • LLM-as-Judge Architectures: Putting Evals Into Your Agent Runtime | Josh Rosen — runtime judge architectures: specialized judge models, decomposed judgment (G-Eval, DAG-based), pairwise comparison, trace-level judging, multi-judge panels (MAJ-EVAL), meta-evaluation benchmarks (LLMBar, JudgeBench, RewardBench, Bloom), hybrid deterministic+LLM control flow (OpenAI grader architecture)