Large Language Model Evaluation

Large language model evaluation is the practice of measuring how well a model or an application built on one performs against defined criteria — accuracy, safety, tone, cost, latency, and task success among them. It exists because generative models don't fail the way traditional software does. A function either returns the right value or it doesn't; a language model can be right, wrong, partially right, right but badly phrased, or confidently wrong in a way no unit test catches.

Evaluation is what turns "the demo looked good" into a measurable, repeatable judgment you can act on before and after shipping.

Why LLM evaluation is hard

Three properties of generative models break the testing habits most engineers already have.

• No single correct output. A summarization or chat response can be phrased many valid ways. There is rarely one string to assert equality against.
• Non-determinism. The same input can produce different outputs across calls, especially at nonzero sampling temperature, so a single pass/fail run tells you less than it would for deterministic code.
• Multidimensional quality. A response can be factually correct but off-tone, or on-topic but unsafe, or fast but wrong. One metric rarely captures whether an output is actually good enough to ship.

These properties don't make evaluation optional — they make it a distinct discipline, with its own methods, rather than an extension of unit testing.

Benchmarks versus application evaluation

Two different activities both get called "LLM evaluation," and mixing them up leads to the wrong tool for the job.

Benchmarks measure general model capability on fixed, public test suites — academic exams, coding problems, reasoning puzzles — so you can compare one model against another. Stanford's HELM (Holistic Evaluation of Language Models) is a widely used framework for this: it defines a broad taxonomy of scenarios and metrics — accuracy, robustness, fairness, efficiency, and more — specifically because narrow, single-number leaderboards hide how a model behaves outside the benchmark's own distribution. Benchmarks answer "which model is generally capable?"

Application evaluation measures whether a specific system — your prompt, your retrieval pipeline, your agent — performs its actual job on your data. A model that leads every public leaderboard can still fail your customer-support use case if your prompts, tools, or edge cases don't resemble what the benchmark tested. Application evaluation answers "does this system do what I need it to do?"

Model selection draws on benchmarks. Shipping decisions draw on application evaluation. You need both, but for different questions.

Evaluation methods

Every major LLM platform — OpenAI, Anthropic, Google Cloud, and Microsoft — converges on the same three families of grading method, because the underlying problem (grading open-ended text) is the same regardless of vendor.

Metric-based (computation) grading. Deterministic, code-computed scores that require no model inference: exact match and string comparison for classification-style tasks with one correct answer, ROUGE and BLEU for measuring overlap with a reference summary or translation, cosine similarity between embeddings for measuring semantic closeness. Fast, cheap, and reproducible, but blind to correct answers phrased differently from the reference.

Human evaluation. People read outputs and score them against a rubric. It is the highest-fidelity signal available, particularly for subjective qualities like tone, helpfulness, or nuanced safety judgments, but it is slow and expensive to run at volume, which makes it a poor fit for evaluating every change on every commit.

LLM-as-judge (model-based) grading. A separate model — ideally not the one being evaluated — scores or compares outputs against a rubric. Google Cloud's Vertex AI Gen AI evaluation service and Microsoft Foundry both ship model-based metrics alongside their computation-based ones, and Anthropic's guidance treats LLM-graded evaluation as one of four practical grading options (multiple-choice, exact match, code-graded, and LLM-graded) precisely because it scales to volumes human review can't reach. Common patterns:

• Pointwise / single-answer grading — score one output in isolation, often on a Likert scale (for example, 1–5 for tone or coherence) or as a binary classification (does this response contain personal information: yes or no).
• Pairwise comparison — show the judge two outputs and ask which is better; useful for A/B testing prompt or model changes.
• Reference-guided grading — score an output against a known-good answer, combining the consistency of metric-based grading with the nuance of a model judge.

LLM-as-judge trades some precision for reach: it needs its own calibration against human judgment before you trust it, but once calibrated it lets you run evaluations continuously, at a scale exact human review cannot match.

Designing an evaluation

Anthropic's guidance on building evaluations frames good success criteria as specific, measurable, achievable, and relevant — not "the model should be good," but concrete targets like "task fidelity F1 score at or above 0.85" or "fewer than 0.1% of responses flagged for toxicity across 10,000 trials." OpenAI's evaluation guidance describes the same discipline as a five-step loop: define success criteria, collect a representative dataset, specify metrics aligned to those criteria, run and compare evaluations, and keep evaluating continuously as the system changes — not once at launch.

Two design habits matter more than any specific metric choice:

• Mirror your real task distribution. An evaluation set built only from clean, easy examples will not catch the failures that matter. Include edge cases: ambiguous requests, adversarial or harmful input, unusually long context, and inputs a human would also find hard to judge.
• Automate what you can, and let volume do the work. A thousand automatically graded test cases with clear pass/fail criteria surface more regressions than a hundred hand-graded ones, because volume gives you statistical confidence and edge-case coverage that a small hand-curated set cannot. Reserve human review for the dimensions that genuinely need human judgment, and use it to calibrate your automated graders rather than replace them.

Avoid what OpenAI's guidance calls "vibe-based evals" — shipping on the strength of a few manually reviewed examples that felt right. It scales to nothing and catches nothing on the next change.

Evaluating RAG and agentic systems

Retrieval-augmented and agentic systems need evaluation dimensions a plain chat model doesn't. Microsoft Foundry's built-in evaluators include RAG-specific metrics — groundedness (is the answer actually supported by the retrieved context) and relevance (did retrieval surface the right material) — measured separately from generation quality, since a system can retrieve well and generate poorly, or the reverse. For agents, both Microsoft Foundry and Google Cloud's Vertex AI Gen AI evaluation service add agent-specific metrics: tool call accuracy (did the agent pick the right tool, with the right arguments, in the right order), task adherence or task completion (did the agent actually finish what it was asked), and intent resolution (did it correctly understand the request in the first place).

This course covers evaluation as a discipline in general. For the retrieval and agent mechanics being evaluated, see the retrieval-augmented generation and AI agents courses in this catalog.

Evaluation in production

Evaluation doesn't stop at deployment. Microsoft Foundry, Google Cloud, and OpenAI all frame it as continuous: run evaluations before deployment to validate behavior, then keep sampling and scoring production traffic afterward to catch drift, since a system that passed evaluation at launch can degrade as usage patterns shift or an upstream model version changes. Production evaluation typically tracks the same quality metrics defined during development alongside operational signals like latency, cost, and error rate, feeding back into the next round of evaluation design rather than treating evaluation as a one-time gate.

Where this course stops

This is an orientation to why LLM evaluation is a distinct discipline, the vocabulary you need to read vendor evaluation documentation, and the main design decisions: which grading method fits which question, and how to build an evaluation set that reflects real use. It does not walk through a specific vendor's evaluation SDK, a specific benchmark's full scenario list, or how to build a custom LLM-as-judge prompt end to end. Those depend on your platform and your application, and the links in this course point to where to go deeper.