Quick answer: LLM evaluation is how you measure whether a model's output is correct, grounded, and relevant, so you can ship changes with evidence instead of vibes. You build a golden set of representative inputs with expected outcomes, score outputs on metrics that fit the task, and combine automated checks, an LLM-as-judge for fuzzy quality, and human review for the hard cases. The goal is a repeatable eval you run in CI, not a one-time spot check.
Why Eval Is Different for LLMs
Traditional software is deterministic: given an input, you assert an exact output. LLMs are not. The same prompt can produce different wording every time, and there is rarely a single correct string. "Paris is the capital of France" and "The capital of France is Paris" are both right, so a naive string comparison fails.
That is why LLM evaluation leans on graded, semantic, and probabilistic checks rather than exact assertions. You are measuring qualities like correctness and groundedness across many examples, then tracking whether those scores go up or down as you change prompts, models, or retrieval.
Without an eval, you are flying blind. You tweak a prompt, it looks better on the three examples you tried, and you have no idea whether you just broke fifty other cases. An eval turns "seems better" into a number you can trust.
The Metrics That Matter
Which metrics you use depends on the task, but a few show up in almost every LLM and RAG evaluation.
| Metric | Question it answers | Where it applies |
|---|---|---|
| Accuracy / correctness | Is the answer factually right? | Any task with a known answer |
| Groundedness / faithfulness | Is the answer supported by the sources? | RAG, summarization |
| Relevance | Does it actually address the query? | Search, QA, RAG |
| Context precision | Did retrieval surface the right documents? | RAG |
| Completeness | Did it cover everything it should? | Summaries, reports |
| Format compliance | Does it match the required schema? | Structured output |
| Safety / tone | Is it appropriate and on-brand? | Anything user-facing |
For RAG specifically, groundedness is the one people skip and regret. A fluent answer that is not actually supported by the retrieved sources is a hallucination in a nice suit. Measuring whether each claim traces back to a source is often more important than measuring raw accuracy.
Method 1: Golden Sets
The foundation of any eval is a golden set: a curated collection of representative inputs paired with expected outputs or expected qualities. It is your test suite for the model.
A good golden set is small enough to run often (dozens to a few hundred cases) and deliberately covers the range of real usage: common cases, edge cases, tricky phrasings, and known failure modes. Do not just sample easy questions. The value is in the hard ones.
For tasks with clear right answers, like classification or extraction, you can score against the golden set with plain assertions:
def evaluate_extraction(model, golden_set):
results = []
for case in golden_set:
output = model(case["input"])
correct = output["category"] == case["expected_category"]
results.append(correct)
accuracy = sum(results) / len(results)
return accuracy
That covers the deterministic slice. The fuzzy slice needs a different tool.
Method 2: LLM-as-Judge (With Caveats)
For open-ended outputs where there is no exact answer, one practical approach is to use a strong model to grade the output against a rubric. Give the judge the question, the answer, the source material, and clear criteria, and ask for a score with a reason.
def judge_groundedness(question, answer, sources):
prompt = f"""Score whether the ANSWER is fully supported by the SOURCES.
Return JSON: {{"grounded": true/false, "reason": "..."}}.
A claim not present in the sources means grounded=false.
QUESTION: {question}
SOURCES: {sources}
ANSWER: {answer}"""
verdict = judge_model(prompt) # a capable model, low temperature
return verdict
LLM-as-judge is useful and scalable, but treat its scores with healthy skepticism. The real caveats:
- Judges have biases. They can favor longer answers, prefer their own writing style, and be swayed by confident tone over correctness.
- They are not ground truth. A judge is an estimate, not an oracle. Validate it against human labels on a sample before you trust it.
- Position and phrasing effects. When comparing two answers, the order you present them can change the verdict. Randomize it.
- Use a rubric, not a vibe. Vague prompts like "is this good?" give noisy scores. Specific, binary criteria are far more consistent.
Used carefully, an LLM judge is a good way to scale evaluation of subjective quality. Used carelessly, it just launders your assumptions into a number.
Method 3: Human Review
Automated metrics and LLM judges get you most of the way, but humans stay in the loop for the cases that matter most. Human review is where you calibrate everything else.
Use it to label a sample that you then use to validate your automated scores, to audit the disagreements where the judge and the assertions conflict, and to review high-stakes outputs before they reach users. You do not need humans on every case. You need them on enough cases to trust the machine on the rest.
Running Evals in CI
An eval you run once is a report. An eval you run on every change is a safety net. The move is to wire your golden set into continuous integration so that a prompt edit, a model swap, or a retrieval change automatically gets scored.
# Fails the build if quality regresses
def test_rag_quality():
scores = run_eval(golden_set)
assert scores["groundedness"] >= 0.90
assert scores["relevance"] >= 0.85
assert scores["accuracy"] >= 0.88
Set thresholds based on your current baseline, then treat a drop as a failing test. This is what lets a team iterate on prompts and retrieval with confidence: every change is measured against the same bar before it ships.
Avoid Overfitting to a Benchmark
The biggest failure mode in LLM evaluation is optimizing for the eval instead of for reality. If you tune your prompts until they ace your golden set, you may just be memorizing that specific set rather than improving the system.
A few guards against it:
- Keep a holdout. Reserve cases you never look at while iterating, and check them occasionally to catch overfitting.
- Refresh the set. Add new real-world failures as they appear in production, so the eval keeps pace with actual usage.
- Do not trust public benchmarks alone. A model topping a leaderboard tells you it is good at that leaderboard. Your task is not that leaderboard. Always evaluate on your own data, which is the same argument we make in how to choose an LLM.
The whole point of an eval is to predict real-world quality. The moment you start gaming it, it stops doing that job.
Where Clean Data Fits
For RAG evaluation, groundedness scores depend on the sources being clean in the first place. If your retrieved context is full of scraped HTML noise, both the model and your judge have a harder time, and your scores get muddier for reasons that have nothing to do with the prompt.
Feeding your pipeline clean text removes that confound. link.sc returns any URL as clean markdown, so the content you index, retrieve, and evaluate against is actual content, not page furniture. Better inputs make your eval measure what you think it measures.
The Bottom Line
Good LLM evaluation combines a representative golden set, metrics that fit the task (with groundedness front and center for RAG), automated scoring, a carefully validated LLM judge, and human review on the cases that matter. Run it in CI so every change is measured, keep a holdout so you do not overfit, and always test on your own data rather than a public benchmark. Evals turn shipping an LLM feature from guesswork into engineering.
Cleaner sources make cleaner evals. Fetch model-ready content for your RAG pipeline with link.sc.