Building an LLM-powered product means answering one question that never goes away: is it actually working?
Metrics like accuracy and F1 score are built for problems with a single correct answer. Most large language model outputs don’t have that. A summary can be factually accurate but miss the point. A chatbot response can be technically correct but unhelpful. A RAG answer can be fluent and completely hallucinated.
This is the problem LLM-as-a-judge solves. You use a language model to evaluate the outputs of another language model, applying whatever criteria matter for your application. The technique scales to thousands of responses, runs continuously in production, and can assess qualities that no rule-based check will ever catch.
This guide covers how LLM-as-a-judge works, the types of judges you can build, how to evaluate AI agents, how to write prompts that hold up in production, and what the real limitations are.
What is LLM-as-a-judge?
LLM-as-a-judge is an evaluation approach where you use a language model to score or classify the outputs of another AI system. Instead of comparing outputs to a fixed answer or running a keyword check, you give the judge model an evaluation prompt that defines your criteria, and it returns a score, label, or explanation.
The technique works for any property you can describe in plain language: helpfulness, tone, factual accuracy, presence of PII, hallucination, whether a user’s question was actually answered. You define what “good” looks like once, and the judge applies it consistently at scale.
A simple example: you have a customer support chatbot. You want to know whether it is resolving user issues. You cannot determine this from token overlap or semantic similarity scores. But you can pass the conversation to a judge model with the instruction, “Did the assistant fully resolve the user’s request? Return ‘Resolved’ or ‘Not resolved’.” Run that across your entire conversation log and you have a metric that maps to what you actually care about.
LLM-as-a-judge is not a fixed metric like accuracy or NDCG. It is a technique for building custom proxy metrics using natural language. The quality of the judge depends on your prompt, your choice of model, and how well you validate against your own expectations.
Why does LLM-as-a-judge work?
The first question practitioners ask is: why would an LLM be any good at catching mistakes made by another LLM?
The answer is that evaluation and generation are fundamentally different tasks. When your product LLM generates a response, it is juggling many inputs simultaneously: a user query, a system prompt, retrieved context, conversation history, and multiple simultaneous constraints. That complexity creates surface area for error.
When a judge LLM evaluates that same response, it is doing something far simpler. It has a focused, single-criterion task: “Is this answer faithful to the context provided? Yes or no.” There is no generation required. The model is acting as a classifier, a task that is generally easier and more reliable than generation.
The broader principle is that it is easier to critique than to create. A musician who struggles to improvise can still reliably tell you whether a note is off-pitch. A copy editor who could not write the article can catch every passive voice construction in it. LLM judges operate on the same asymmetry.
There is also the question of independence. When a malicious user triggers a prompt injection attack that causes your chatbot to generate problematic content, the conversation prompt and history are contaminated. An external judge model receives only the output and the evaluation prompt. It has no knowledge of the adversarial input. It can still evaluate the output on its merits.
This does not mean LLM judges are better than the systems they evaluate. They are asked to do a simpler, more focused task, and they are reliably good at it when that task is well-defined.
Types of LLM judges
There are three main patterns for LLM-as-a-judge, suited to different evaluation scenarios.
Pairwise comparison
In pairwise comparison, the judge sees two responses to the same prompt and decides which is better. This approach was formalized in the paper “Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena” (Zheng et al., 2023), where GPT-4 was used as an evaluator against crowdsourced human preferences. Agreement exceeded 80%, comparable to the agreement rate between different human evaluators.
Pairwise comparison is most useful during development, when you are comparing two models, two prompt versions, or two retrieval strategies. You generate both outputs for the same input and ask the judge to pick the better one.
The main limitation is that pairwise evaluation requires generating two outputs for every input. It is a development-phase tool. You cannot run it on live traffic without also generating an alternative response to compare against.
Evaluation by criteria
Direct scoring evaluates a single response against criteria you define. This is the most common pattern for production monitoring because it does not require a second response.
You can evaluate any property you can describe: conciseness, politeness, tone, relevance, whether the response contains PII, whether an agent refused a task it should have completed, whether the user repeated their question (a signal that the previous answer failed them).
Binary classifications tend to be more reliable than continuous scales. “Helpful” vs “Unhelpful” produces more consistent results than a 1-to-5 helpfulness score, because the difference between a 3 and a 4 is difficult to define clearly enough for an LLM to apply consistently.
You can also evaluate full conversation transcripts rather than individual responses. As long as the conversation fits within the judge model’s context window, this works the same way. Conversation-level evaluations are useful for:
- Detecting unresolved sessions: did the user’s issue get solved by the end?
- Identifying frustration signals: did the user express negative emotions or repeat themselves?
- Catching denial patterns: did the agent refuse to complete a valid task at any point?
Reference-based evaluation
Reference-based evaluation provides additional context alongside the response being judged, which is necessary for evaluation tasks that require outside information to make a determination.
Correctness against a reference answer. You provide a “golden” reference answer and ask the judge whether the generated response conveys the same meaning, even if the wording differs. This is a practical alternative to ROUGE or BLEU, which measure word overlap and miss paraphrases that are semantically equivalent.
RAG context faithfulness. In retrieval-augmented generation systems, the model generates answers from retrieved documents. The judge receives both the generated answer and the source documents and evaluates whether the answer is grounded in the retrieved context, or whether the model invented details not present in the source. A faithfulness judge catches hallucinations that a generic accuracy check would miss.
Question-answer relevance. Even without a reference answer, you can evaluate whether a response actually addresses the question asked. This is valuable for live monitoring where you have no ground truth to compare against.
Evaluating AI agents with LLM judges
Standard LLM evaluation measures whether a response is good. Agent evaluation measures whether a multi-step behavior is good, which requires a different approach.
AI agents fail in ways that single-response evaluation does not catch. An agent might produce a fluent final answer while calling the wrong tool, using incorrect parameters, or taking three unnecessary steps to get there. The answer looks fine. The execution was broken.
LLM judges can evaluate each dimension of agent behavior separately:
Agent planning. Before the agent acts, does it have a sound plan? Does the plan use only valid tools? Is it the most efficient path to the goal, or does it include redundant steps?
Tool selection. Did the agent choose the right tool for the task? Are there more appropriate tools it overlooked?
Parameter extraction. Did the agent correctly extract the parameters required for tool execution? Are they formatted correctly? Are any required parameters missing?
Tool calling. Is the tool invocation correctly structured? Are the parameters accurate and complete?
Path evaluation. Does the agent follow a logical, efficient sequence? Does it avoid loops and dead-ends?
Task completion. At the end of the workflow, was the user’s objective actually achieved?
The key architectural decision for agentic AI evaluation is whether to evaluate at the span level (individual tool calls and model outputs within a workflow) or the trace level (the entire end-to-end workflow). Span-level evaluation lets you pinpoint where failures occur. Trace-level evaluation tells you whether the overall task was completed correctly. Both matter: a workflow can succeed overall while individual steps are inefficient, and a workflow can fail at the last step after a sequence of correct decisions.
Agent evaluation is more complex to set up than single-response evaluation, but the investment pays out in proportion to the autonomy your agents have. The longer and more independent an agent session is, the more surface area exists for failures that span-level checks will not surface.
Offline vs online evaluation
LLM judges can run in two modes, and the distinction shapes how you design your evaluation infrastructure.
Offline evaluation happens outside of live traffic, typically on a fixed dataset. You use it to compare models or prompt versions, run regression tests before deploying changes, and validate that a new configuration is better than the old one before shipping it. Offline evaluation supports complex, multi-step analysis with no latency constraints.
Online evaluation runs on live production traffic. You sample a percentage of real interactions, pass them through the judge, and track metrics over time on a dashboard. This is how you detect degradation: a spike in hallucination rate, a drop in task completion, an increase in user frustration signals. Online evaluation requires more careful attention to latency and cost because it runs continuously.
The strongest evaluation infrastructure uses both. You validate changes offline before shipping them, using the same judge prompts you will run in production. This means your pre-launch and post-launch scores are directly comparable. Catching a regression in production is valuable. Catching it in offline testing before it ships is better.
Teams that build their evaluation framework early, around their actual failure modes, carry the same rubric through development, staging, and production. Teams that build it after something breaks in production are starting from scratch with pressure on.
How to create an LLM judge
Building an LLM judge is a small ML project with a structured process. The quality of the judge depends on the upfront work you put in.
Step 1: Define the evaluation scenario precisely. What exactly should the judge assess? Tone? Correctness? Task completion? Keeping the evaluation to a single criterion produces more reliable results than asking the judge to simultaneously evaluate multiple properties. If you need to assess several dimensions, create separate evaluators for each and combine results deterministically afterward.
Step 2: Prepare an evaluation dataset. Collect examples from your system’s actual outputs, or generate synthetic examples that represent the range of inputs you expect. The dataset does not need to be large, but it should include challenging cases and not just easy ones.
Step 3: Label the dataset manually. This is the most valuable step and the most commonly skipped. Label the examples yourself, according to the criteria you defined in Step 1. The process of labeling forces you to discover edge cases and ambiguities in your definition. Any clarification you need to make a judgment call is a clarification that belongs in your evaluation prompt.
Research by Shankar et al. (2024) in “Who Validates the Validators?” describes this as criteria drift: your understanding of what “good” means evolves as you see actual model outputs. Starting with manual labels anchors your criteria before that drift begins.
Step 4: Write the evaluation prompt. Draw on the decisions you made while labeling. Define each possible label explicitly. If a case is ambiguous, specify which direction to err. Include examples if the distinction between labels is nuanced.
Step 5: Evaluate and iterate. Run the judge on your labeled dataset and measure agreement with your labels. For binary evaluations, precision and recall against the class you care most about catching are useful measures. If the judge misclassifies cases, examine the failures: are the instructions unclear? Does the judge need examples? Is the criterion too ambiguous to resolve without splitting it into two separate evaluators?
Domain experts, including non-technical team members, play a meaningful role in the label definition and prompt review stages. Since evaluation prompts are written in plain language, product managers and subject matter experts can review and improve them without touching code.
How to write evaluation prompts
The evaluation prompt is the core of any LLM judge. Small changes in phrasing produce measurably different results, and what works on one model may behave differently on another.
Use binary or low-precision scoring. Ask for “Helpful” vs “Unhelpful” before you ask for a 1-to-5 scale. Binary labels produce more consistent results because they force a clear decision. If a two-option label is too coarse, a three-option scale (“Relevant,” “Partially Relevant,” “Irrelevant”) is still more reliable than five or ten options. If you cannot clearly define the difference between adjacent scores on your scale, the model cannot either.
Define each label explicitly. Do not assume the model knows what you mean by “toxic” or “helpful” or “correct.” Write out what each label means. If you want the judge to flag borderline cases, say so: “If unsure, err toward flagging.” Without explicit guidance, the model falls back on distributions from training data that may not match your use case at all.
Split complex criteria into separate evaluators. An evaluation prompt asking the judge to simultaneously assess accuracy, completeness, tone, and relevance is asking four questions at once. Each of those four dimensions should have its own evaluator. You can combine the results afterward, such as flagging a response if any single criterion fails, assigning each a weight, or summing binary scores for an overall quality index.
Add examples for nuanced distinctions. When the difference between labels is subtle, examples make the distinction concrete. This is few-shot prompting applied to evaluation. Be careful about the balance and ordering of examples: a skewed example set or examples that cluster at one end of the list can bias the judge toward those patterns. Research by Zhao et al. (2021) in “Calibrate Before Use” showed that few-shot performance varies significantly based on example order and format.
Encourage step-by-step reasoning. Asking the judge to explain its reasoning before returning a final label, commonly called chain-of-thought prompting (Wei et al., 2022), improves both evaluation quality and interpretability. When a judge flags a response as hallucinated or unhelpful, the reasoning trace shows exactly what triggered the label. Chain-of-thought works best when the reasoning appears before the verdict. The explanation informs the label rather than the model generating a label and then rationalizing it.
Set a low temperature. LLM evaluation does not benefit from creative variation. A low temperature keeps outputs consistent for the same input, which is what you need when tracking trends over time.
Use a capable model. More capable models align more reliably with human judgment. Start with a strong model to establish your baseline. Once the prompt is validated, you can test whether smaller or cheaper models meet your quality threshold for that specific task.
Request structured output. JSON output makes evaluation results easy to parse, aggregate, and feed into downstream analysis. Define the exact output schema in your prompt.
Running LLM judges in production
Offline evaluation validates your system before launch. Production evaluation tells you whether it stays healthy after.
The production monitoring workflow has three components.
Tracing. Instrument your AI application to log all inputs, model calls, and outputs. Once you have this data, you can review individual interactions, look for patterns, and feed samples into your judge. Without tracing, you are evaluating a system you cannot observe. Connecting tracing to MLOps workflows makes this data actionable beyond individual debugging sessions.
Scheduled evaluations. Set up regular evaluation jobs that run samples of live traffic through your LLM judges. Depending on your traffic volume, you might evaluate 5-10% of interactions rather than every one. Running judges on every request in real time adds latency and cost. Scheduled batch evaluation on recent traffic gives you the trend data you need at a fraction of the cost.
Combine LLM judges with cheaper methods where possible. A regex check for keywords like “I can’t help with that” can flag agent refusals without an LLM call. Reserve LLM evaluation for criteria that simpler methods cannot handle.
Dashboards and alerts. Track evaluation metrics over time, not just as one-off snapshots. A 72% helpful rate on its own tells you little. A drop from 88% to 72% over three days tells you something changed. Set thresholds for alerts so that significant degradation triggers notification before it compounds.
When a metric degrades, trace back to the individual conversations or responses driving the change. Export failing cases as test data for your evaluation dataset. This closes the loop: production issues feed back into offline evaluation, which validates fixes before they ship.
AI drift is one of the primary reasons continuous evaluation matters. Model behavior can shift as underlying model weights are updated, retrieval indexes age, or the distribution of user inputs changes. A judge that ran consistently for three months can start flagging new failure patterns if any of those conditions change.
Biases in LLM judges and how to handle them
LLM judges are not neutral arbiters. They carry biases from training data and from the structure of evaluation prompts, and those biases affect results in predictable ways.
Position bias is the tendency to favor responses based on their position in the prompt. In pairwise evaluations, judges often prefer whichever response appears first or last. The fix is to evaluate both orderings and average the results, or to move to direct scoring instead of pairwise comparison when the bias risk is high.
Verbosity bias is the tendency to prefer longer responses, even when the shorter one is more accurate or relevant. If your evaluation criteria do not explicitly address length, a judge may systematically reward verbosity. Counter this by including conciseness as an explicit criterion or by testing your judge against cases where the correct shorter answer should win.
Self-enhancement bias is the tendency for a model to favor outputs generated by the same model family. When using a model from one provider to evaluate outputs from that same provider’s model, this is worth testing. Using a judge model from a different provider eliminates this concern entirely.
Research by Verga et al. (2024) in “Replacing Judges with Juries” addresses this problem by running multiple judge models and combining their verdicts, similar to an ensemble approach. The paper found that a panel of smaller models outperformed a single large judge while reducing intra-model bias and costing over seven times less.
Beyond these three well-documented biases, under-specified prompts introduce a subtler form of inconsistency. When the same input produces different labels across runs, the evaluation criterion is likely ambiguous. Adding explicit definitions, tightening the prompt language, or splitting the criterion into two distinct evaluators will reduce variance.
Pros and cons of LLM-as-a-judge
Why LLM judges work well
No reference answer required. You can evaluate properties of live responses without knowing in advance what the correct answer should have been. This is the main reason LLM judges are useful for production monitoring, where ground truth is rarely available.
Flexibility. You can evaluate any property you can describe. Adding a new evaluation criterion means writing a new prompt, not retraining a model.
Scale. Once configured, a judge runs thousands of evaluations per hour at a fraction of the cost of human review.
Accessibility. Because evaluation criteria are written in natural language, domain experts without machine learning backgrounds can participate in defining, reviewing, and improving judges.
Coverage across languages. LLM judges extend naturally to multilingual applications. A human reviewer team would require language-specific hiring. A well-prompted judge model handles language variation without additional setup.
Where LLM judges fall short
Imperfect consistency. Even well-designed judges produce inconsistent results for edge cases, especially when criteria are complex or the distinction between labels is subtle. Running multiple evaluations and aggregating results reduces variance but adds cost.
Bias. Position bias, verbosity bias, and self-enhancement bias are documented and manageable, but managing them requires deliberate prompt design and testing. Unmanaged, they produce results that look reliable but measure the wrong thing.
Cost at scale. Running LLM judges on every response in real time is expensive. The cost scales with model capability and call volume. Sampling strategies and combining LLM judges with cheaper rule-based checks reduce this, but it remains a real constraint.
Data privacy risk. If you use a third-party LLM API for evaluation, your response data leaves your infrastructure. For applications handling sensitive information, this requires careful review. On-premise or self-hosted judge models eliminate the external API dependency but add operational complexity.
Setup investment. A reliable LLM judge requires defining criteria, preparing a labeled dataset, iterating on the evaluation prompt, and building the monitoring infrastructure. It is not a tool you deploy once and forget.
Alternatives to LLM-as-a-judge
LLM judges are the right tool for evaluating nuanced, semantic properties of text at scale. They are not always the right tool.
Human evaluation is still the gold standard for tasks that require contextual judgment, domain expertise, or cultural sensitivity that a language model may not have. Human review does not scale, but it should be the anchor for validating that your LLM judges are calibrated correctly. Reserve human evaluation for setting up label definitions, reviewing judge failures, and sampling a percentage of production traffic.
Rule-based checks are faster, cheaper, and deterministic. Checking for the presence of specific keywords, validating that output conforms to a schema, or confirming that a required field is populated requires no LLM call. Use rule-based checks for everything they can handle, and reserve LLM judges for what they cannot.
Embedding-based similarity measures how semantically similar a generated response is to a reference answer, using a pre-trained embedding model rather than an LLM. This is useful for regression testing, where you want to confirm that updated outputs are not drifting from a known-good baseline. It is less useful than an LLM judge when you need to evaluate qualities that are not captured by semantic distance alone.
ROUGE and BLEU measure n-gram overlap between a generated output and a reference text. They work for structured generation tasks like summarization or translation where there are accepted reference outputs. They are unreliable for open-ended generation where multiple correct phrasings exist.
Fine-tuned evaluator models are worth considering once you have accumulated a labeled dataset large enough to train on. A fine-tuned model for a specific evaluation task can be faster and cheaper to run than a general-purpose LLM judge, and it does not require an API call to an external provider. The limitation is rigidity: a fine-tuned evaluator does not generalize to new criteria without retraining.
User feedback signals are the most direct signal available. Thumbs up/down ratings, follow-up questions, explicit complaints, and task completion rates measure what users actually experience. Combine user feedback signals with LLM judge evaluations: the LLM judge catches what users do not explicitly flag, and user feedback validates whether the judge’s criteria reflect what users actually care about.
In practice, reliable evaluation infrastructure combines multiple methods. Rule-based checks handle deterministic criteria cheaply. LLM judges handle semantic and subjective criteria at scale. Human review validates both and catches what automated methods miss.
Frequently asked questions
When should I use LLM-as-a-judge evaluation? Use LLM judges when you need to evaluate properties that cannot be determined by rule-based checks or metric calculations, and when human review at the volume you need would be too slow or expensive. Common triggers: you are building a chatbot and cannot determine whether responses are helpful without reading them; you are running a RAG system and need to detect hallucinations at scale; you are monitoring an AI agent and need to evaluate task completion across thousands of sessions.
How much does LLM-as-a-judge cost? Cost depends on the model you use, the length of your evaluation prompts, and the volume of evaluations. Sampling strategies, where you evaluate 5-10% of traffic instead of 100%, reduce this proportionally. Smaller or self-hosted models bring the per-evaluation cost down significantly if your quality thresholds allow it.
What are common pitfalls with LLM-as-a-judge? The most common failure modes are: leaving evaluation criteria vague and assuming the model interprets them the way you intend, not validating the judge against manually labeled ground truth before deploying it, and treating 100% pass rates as a sign of success rather than a sign that the evaluation criteria are not challenging enough.
Can I use LLM judges for AI agent evaluation? Yes, and it is worth doing deliberately. Agent evaluation requires breaking behavior down into distinct dimensions: planning quality, tool selection, parameter extraction, path efficiency, and task completion. Each dimension benefits from its own judge with criteria specific to that stage of the workflow. Span-level evaluation catches failures within individual steps; trace-level evaluation tells you whether the end-to-end task was completed.
What models work best as LLM judges? More capable models produce evaluations that align more closely with human judgment. Start with a frontier model to establish your baseline and validate your prompts. Once you have a validated judge, test whether a smaller or cheaper model achieves acceptable agreement rates on your specific evaluation task. Research by Verga et al. (2024) found that a panel of smaller models can outperform a single large judge while reducing costs significantly.
If you are building AI systems that use LLMs in production, evaluating output quality is one side of the reliability question. Understanding the security risks that LLMs introduce is the other side. Both disciplines inform each other more than they appear to at first.