Large Language Model Evaluation
LLM evaluation measures how well a large language model performs on specific tasks through benchmarks, human judgments, and automated metrics. It assesses accuracy, safety, reasoning, hallucination rates, and task-specific quality to inform model selection and improvement.
itArtificial intelligence and machine learning | OpenSkills.info
Course pathWalk it in order
Look it upDip in anytime
Go furtherLeaves this page
Don't Panic
Don't Panic - Large Language Model Evaluation
Large language model evaluation answers a practical question: is this model, prompt, retrieval pipeline, or agent good enough to ship? It is how you avoid finding the wrong answer, unsafe answer, expensive answer, and beautifully phrased useless answer after somebody else has read it.
Language-model output is not a calculator result. There is often no single correct sentence. The same input can produce different results across calls. Quality has several axes: accuracy, safety, tone, latency, cost, and task success. One score is a suspiciously small suitcase for that luggage.
A benchmark compares general capability on fixed public tasks. It helps with model selection. Application evaluation asks whether a specific system handles real prompts, data, edge cases, tools, and users. It informs shipping decisions. A leaderboard winner can still make a poor support assistant if the benchmark and the job have little in common. The leaderboard is answering a different question.
The practical loop is unromantic. Define success criteria. Collect a representative dataset. Choose metrics that match the criteria. Run and compare the evaluation. Keep evaluating as the system changes. Polished examples are not representative traffic, and a few answers that felt right are a vibe-based eval: a confidence ceremony with limited diagnostic value.
Metric-based checks are fast and repeatable when the requirement is exact. Human review handles nuanced quality but does not scale cheerfully. LLM-as-judge grading lets a separate model apply a rubric to open-ended output, including pointwise, pairwise, and reference-guided comparisons. It earns trust only after calibration against human judgment; a judge is a measurement instrument, not an oracle in a trench coat.
RAG and agents add traps. Groundedness asks whether an answer is supported by retrieved context; relevance asks whether retrieval found the right material. Agents also need intent resolution, tool call accuracy, and task adherence. Measure them separately, because a system can retrieve well and answer poorly, or select the right tool and use the wrong arguments.
Read the Intro for the map, Slides for relationships, and Cheatsheet for working terms. The practice reference turns a production failure into a repeatable test. Make the next change measurable before it gets theatrical.
Where this skill leads
Relevant careers
See how this topic contributes to broader role-level skill maps.
Sources
- https://developers.openai.com/api/docs/guides/evaluation-best-practices
Supports
- Evaluations are a continuous process, not a one-time launch activity
- Five-step evaluation workflow: define success criteria, collect representative dataset, specify metrics, run and compare evals, continuously evaluate
- Metric-based evals (exact match, ROUGE/BLEU, function call accuracy) for automated regression testing
- Human evals as highest quality but expensive and slow
- LLM-as-judge subtypes: pairwise comparison, single-answer grading, reference-guided grading
- Evaluation needs vary by architecture: single-turn, workflows, single-agent, multi-agent
- Vibe-based evals and generic academic metrics disconnected from production are anti-patterns
- https://developers.openai.com/api/docs/guides/evals
Supports
- Evals API for configuring evaluations programmatically or via dashboard
- https://github.com/openai/evals
Supports
- Evals is an open-source framework and registry of benchmarks for evaluating LLMs and LLM systems
- https://platform.claude.com/docs/en/docs/test-and-evaluate/develop-tests
Supports
- Success criteria should be specific, measurable, achievable, and relevant
- Evaluation dimensions include task fidelity, consistency, relevance and coherence, tone and style, privacy, context utilization, latency, price
- Grading methods: exact match/string comparison, similarity-based metrics (cosine similarity, ROUGE-L), LLM-based grading (Likert scale, binary classification, ordinal scales)
- Evaluations should mirror real-world task distribution and factor in edge cases
- Prioritize volume with automated grading over fewer hand-graded tests
- Example multidimensional target table (task fidelity F1, safety toxicity rate, error severity, latency)
- https://docs.cloud.google.com/vertex-ai/generative-ai/docs/models/run-evaluation
Supports
- Computation-based evaluation (no model inference) vs. model-based evaluation (judge models)
- Pointwise evaluation (individual outputs) vs. pairwise evaluation (comparing two outputs, AutoSxS)
- General evaluation workflow: define metrics, prepare dataset, run evaluation, interpret results
- https://docs.cloud.google.com/vertex-ai/generative-ai/docs/model-reference/evaluation
Supports
- Gen AI evaluation service lets you evaluate LLMs with custom criteria, returning task-specific metrics
- Metric types include model-based metrics (PointwiseMetric, PairwiseMetric) and in-memory computed metrics (ROUGE, BLEU, tool function-call metrics)
- Batch evaluation for large datasets via asynchronous long-running operations
- Agent evaluation metrics for AI agents built with different frameworks
- https://learn.microsoft.com/en-us/azure/foundry/how-to/evaluate-generative-ai-app
Supports
- Evaluations measure performance, quality, and safety before deployment and to monitor production quality after
- Built-in evaluators cover general-purpose quality (coherence, fluency), RAG-specific metrics (groundedness, relevance), safety/security metrics, and agent-specific metrics (tool call accuracy, task completion)
- Three metric categories: AI Quality (AI-Assisted), AI Quality (NLP), Risk and Safety
- Agentic metrics: Intent Resolution, Tool Call Accuracy, Task Adherence
- Integration with Azure Monitor Application Insights for real-time dashboards tracking operational metrics, token consumption, latency, error rates, quality scores
- https://crfm-helm.readthedocs.io/en/latest/
Supports
- HELM is an open-source Python framework created by Stanford CRFM for holistic, reproducible, transparent evaluation of foundation models
- HELM taxonomizes scenarios (use cases) and metrics (desiderata) rather than relying on a single aggregate score
- HELM evaluates across multiple dimensions including accuracy, fairness, robustness, and efficiency
- https://github.com/stanford-crfm/helm
Supports
- HELM is an open source Python framework for holistic, reproducible, and transparent evaluation of foundation models, including LLMs and multimodal models
- https://arxiv.org/abs/1804.07461
Supports
- GLUE benchmark
- https://arxiv.org/abs/1905.00537
Supports
- SuperGLUE benchmark
- https://arxiv.org/abs/2009.11462
Supports
- RealToxicityPrompts
- https://arxiv.org/abs/2104.14337
Supports
- Dynabench
- https://arxiv.org/abs/2206.04615
Supports
- BIG-bench
- https://arxiv.org/abs/2211.09110
Supports
- HELM
- https://arxiv.org/abs/2306.05685
Supports
- MT-Bench
- Chatbot Arena
- and judge bias
- https://arxiv.org/abs/2406.19314
Supports
- LiveBench
- https://www.braintrust.dev/docs/evaluate
Supports
- Evaluation experiments
- https://docs.langchain.com/langsmith/evaluation-concepts
Supports
- Component evaluation
- https://langfuse.com/docs/evaluation/core-concepts
Supports
- Experiments and online evaluation
- https://arize.com/docs/phoenix/
Supports
- LLM evaluation and tracing
- https://galileo.ai/
Supports
- Production guardrails