LLM Evaluation: Benchmarks, Metrics, Judges and Statistics

Learn how to measure LLM quality honestly: benchmarks and their pitfalls, accuracy, perplexity and pass@k, pairwise comparison and Elo, LLM-as-judge and its biases, and why confidence intervals matter.

28 minIntermediateCode Examples

The Core Idea: You cannot improve what you cannot measure. Good LLM evaluation combines automatic metrics, model judges and human review, built on your own test set, with honest uncertainty ranges.

  1. Beginner: Why Evaluating LLMs Is Hard

For a spam filter, evaluation is easy: count how many emails were labeled correctly. For an LLM that writes essays, answers questions and writes code, there is no single correct answer to compare against, and quality has many sides: correctness, helpfulness, safety, tone and style.

Good evaluation (often shortened to evals) is how you decide which model to use, whether a prompt change helped, and whether an update broke something. Without it you are guessing.

Automatic metricsFast and cheap
Compare the output to a known answer with code.
Model judgesScalable opinions
Another LLM grades the answers.
Human reviewThe gold standard
People rate quality. Slow and expensive but most trusted.

  1. Beginner: Benchmarks and Their Limits

A benchmark is a fixed set of questions with known answers that many models can be tested on, such as multiple-choice exams across school subjects, math word problems or coding challenges. They make models easy to compare.

ProblemWhat it means
ContaminationBenchmark questions leaked into the training data, so the model has partly memorized them
SaturationTop models all score near the maximum, so the test no longer separates them
Narrow coverageA benchmark measures only what it contains, not your real task
GamingModels can be tuned specifically to do well on famous tests

Golden rule: Public benchmarks are a starting point. The evaluation that matters most is a small test set built from your own real use cases.

  1. Medium: Metrics You Can Compute

MetricWhat it measuresUse for
Accuracy / exact matchShare of answers that match the expected answerClassification, short factual answers
PerplexityHow surprised the model is by real text. Lower is betterComparing language models on the same text
pass@kChance that at least one of k tries passes the testsCode generation
ROUGE / BLEUWord overlap with a reference textSummaries and translation (weak for meaning)
Elo / win rateRatings from head-to-head comparisonsOverall chat quality

For code, pass@k is computed from nn samples of which cc are correct:

pass@k=1−(n−ck)(nk)\text{pass@}k = 1 - \frac{\binom{n-c}{k}}{\binom{n}{k}}

Perplexity in one line: It is ee raised to the average negative log-probability of the correct tokens. A perplexity of 3 means the model is as unsure as choosing between 3 equally likely words.

  1. Medium: LLM-as-Judge and Pairwise Comparison

Humans find it much easier to say which of two answers is better than to give a score. Two popular methods build on this. In human or AI pairwise comparisons, a judge sees two answers and picks a winner; many comparisons are combined into Elo-style ratings. In LLM-as-judge, a strong model grades answers using a written rubric, which is cheap and fast.

Judge biasWhat happensFix
Position biasPrefers the answer shown first (or last)Run each pair twice with the order swapped
Length biasPrefers longer answersTell the judge to ignore length, or control for it
Self-preferenceA model rates its own style higherUse a different model as judge
Vague rubricScores are inconsistentWrite a clear rubric with examples

Always check the judge: Compare the judge's scores with a small set of human ratings. If they disagree a lot, the judge cannot be trusted.

  1. Advanced: Statistics and Building an Eval Process

A score of 80% on 50 questions is not the same as 80% on 500 questions. With small samples the true accuracy could be far from what you measured, so report an uncertainty range such as a 95% confidence interval.

HabitWhy
Report confidence intervalsSmall differences between models are often just noise
Use a fixed, versioned test setSo results are comparable over time
Include hard and edge casesAverages hide the failures users notice
Run evals on every prompt or model changeCatch regressions before users do
Keep a hidden setNever tune on it, so it stays honest
Review failures by handNumbers tell you how much, reading outputs tells you why

Remember: A better score is only meaningful if the difference is larger than the uncertainty. Measure honestly, then decide.

  1. Visual Explanation: Judges and Error Bars

The figure compares three kinds of judges, then shows how the same measured score becomes more trustworthy with more test questions.

Part 1 · Three Ways to Judge an AnswerPick by speed vs. trust
AutomaticFast, cheap
Exact match, tests, perplexity
↓
Runs in seconds
↓
Great for closed answers
LLM judgeScalable
Rubric + two answers
↓
Judge picks a winner
↓
Needs bias checks
HumanMost trusted
People compare or score answers
↓
Slow and costly
↓
Gold standard for quality
Part 2 · How Much Can You Trust a Score?More questions, tighter range
80% on 50 questionsWide range
Measured accuracy: 80%
↓
95% interval: about 67% to 89%
↓
True skill is uncertain
80% on 500 questionsNarrow range
Measured accuracy: 80%
↓
95% interval: about 76% to 83%
↓
Much more reliable
Click to see why sample size mattersCompare ▼
With few questions, luck matters a lot. The interval shrinks roughly with the square root of the number of questions, so ten times more questions makes the range about three times narrower. The code below computes these intervals.

Part 1 reads left to right from fastest to most trusted. Part 2 uses real numbers from the code: the same 80% means very different things on 50 and 500 questions. Open the bar for the square-root rule.

  1. Python Implementation: Intervals, pass@k, Elo and Perplexity

This script computes four evaluation tools from scratch: an accuracy confidence interval (Wilson), the pass@k estimator for code, Elo ratings from head-to-head results, and perplexity from token probabilities.

llm_evaluation.pyPython 3.10+ · NumPy and the standard library
import numpy as npfrom math import comb, sqrt # 1. Accuracy with an honest uncertainty range (Wilson 95% interval)def accuracy_with_interval(correct, total, z=1.96):    p = correct / total    centre = (p + z * z / (2 * total)) / (1 + z * z / total)    half = z * sqrt(p * (1 - p) / total + z * z / (4 * total * total)) / (1 + z * z / total)    return p, centre - half, centre + half for total in [50, 500]:    p, lo, hi = accuracy_with_interval(int(0.8 * total), total)    print("80% on", total, "questions -> true accuracy is probably between", round(lo * 100, 1), "% and", round(hi * 100, 1), "%") # 2. pass@k: chance that at least one of k tries solves the problemdef pass_at_k(n, c, k):    if n - c < k:        return 1.0    return 1 - comb(n - c, k) / comb(n, k) print("20 tries, 5 correct -> pass@1:", round(pass_at_k(20, 5, 1), 3), "| pass@5:", round(pass_at_k(20, 5, 5), 3), "| pass@10:", round(pass_at_k(20, 5, 10), 3)) # 3. Pairwise comparisons turned into ratings (Elo)ratings = {"model A": 1000.0, "model B": 1000.0}matches = [("model A", "model B")] * 7 + [("model B", "model A")] * 3          # (winner, loser)for winner, loser in matches:    expected = 1 / (1 + 10 ** ((ratings[loser] - ratings[winner]) / 400))    ratings[winner] += 16 * (1 - expected)    ratings[loser] -= 16 * (1 - expected)print("Ratings after 10 judged matches:", {k: round(v) for k, v in ratings.items()}) # 4. Perplexity from the average log-probability of the correct tokenslog_probs = np.log([0.5, 0.25, 0.8, 0.1])print("Perplexity:", round(float(np.exp(-log_probs.mean())), 2))
EXPECTED OUTPUT (your numbers may differ slightly)
80% on 50 questions -> true accuracy is probably between 67.0 % and 88.8 %
80% on 500 questions -> true accuracy is probably between 76.3 % and 83.3 %
20 tries, 5 correct -> pass@1: 0.25 | pass@5: 0.806 | pass@10: 0.984
Ratings after 10 judged matches: {'model A': 1020, 'model B': 980}
Perplexity: 3.16

The same 80% gives a range from 67% to 89% on 50 questions but only 76% to 83% on 500. With 5 correct out of 20 tries, pass@1 is 0.25 but pass@10 is already 0.98, which is why the number of tries must always be reported.

Interactive Knowledge Check

Question 1: A model scores 80% on only 50 questions. What should you conclude?
Question 2: What is position bias in an LLM judge?

Key Points

✓LLM quality has many sides and often no single right answer, so evaluation needs several methods.
✓Public benchmarks suffer from contamination, saturation and narrow coverage. Build a test set from your real use cases.
✓Use accuracy for closed answers, perplexity for language modeling and pass@k for code.
✓Pairwise comparisons and Elo ratings turn easy judgments (which answer is better?) into rankings.
✓LLM judges are cheap but biased (position, length, self-preference), so check them against human ratings.
✓Report confidence intervals. Differences smaller than the uncertainty are not real.

The Big Picture

Evaluation in One Line

Real test set + the right metric + a checked judge + error bars = trustworthy decisions

Teams that evaluate carefully improve faster, because every change is a measured experiment.

Up Next: Evaluation also shows where today's models fall short. In Emerging LLM Techniques you will see reasoning models, test-time compute, distillation and where the field is heading.