LLM Evaluation Methods

1. What Is an LLM Evaluation Method?

An LLM evaluation method is the mechanism used to decide whether an LLM's output is good or not. In simple terms, it is the actual procedure that takes an output and produces a judgment about it.

For example, an evaluation pipeline may check whether a RAG retriever is retrieving relevant documents. The method determines whether that checking is done by:

• A program
• A human
• Another LLM acting as a judge

2. The Three Primary Evaluation Methods

3. Programmatic / Deterministic Evaluation

Case Study: RAG Retriever for Campus X

Suppose Campus X builds a RAG chatbot. The retriever receives a question and retrieves the most relevant documents from a vector database.

Evaluation target and task

• Target: Retriever component
• Task: Determine whether the retriever retrieves the relevant documents.
• Success criterion: Recall@K

Building the golden dataset

A human expert first creates a dataset of around 50–100 representative user questions. The expert identifies which documents contain the correct information for each question.

Example

Question: “What are the prerequisites for the ML course and how long is it?”

Suppose the correct information exists in documents 1001 and 1003. The retriever is configured with K = 5 and returns:

1001, 1002, 1004, 1005, 1006

Only one of the two relevant documents was retrieved.

4. Recall@K

Recall@K asks:

For the example:

Ideally, recall is 100%. It cannot exceed 100% or fall below 0%.

Dataset-level evaluation

For every question:

1. Send the question to the retriever.
2. Retrieve its top-K documents.
3. Compare retrieved documents with the known relevant documents.
4. Calculate Recall@K for that question.
5. Repeat for all questions.
6. Average the per-question recall values.

How can the retriever be improved?

• Use a better embedding model.
• Perform query expansion using an LLM.
• Increase K, e.g. from 5 to 10.
• Add a reranker.
• Improve chunking and retrieval strategy.

Other retrieval metrics can also be considered, such as precision and ranking-related metrics. Recall@K is used here to keep the example simple.

5. Human Evaluation

Case Study: Helpfulness of a Campus X Chatbot

Now consider a general Campus X chatbot that answers questions such as course duration, fees, certificates, validity, refunds, and installments.

The target is the entire application, not just one component.

Task

Evaluate the helpfulness of chatbot answers.

Helpfulness can include:

• Accuracy
• Completeness
• Appropriate tone
• Usefulness to the user

Why not a simple programmatic metric?

Helpfulness is nuanced. There may be no single deterministic formula that can reliably determine whether an answer is helpful.

Example rubric: 1–5

Human evaluation flow

1. Create a representative dataset of around 50–100 questions.
2. Send each question to the chatbot.
3. Collect the generated answer.
4. Give the question, answer, and rubric to a human evaluator.
5. The evaluator assigns a score.
6. Average the scores to obtain an overall helpfulness score.

6. Types of Human Evaluation

1. Direct Grading / Rating

A human reads an output and assigns a score based on a predefined rubric.

2. Red Teaming

A group of humans deliberately attacks an LLM-based system to discover weaknesses, safety failures, jailbreaks, or other ways the system can break.

3. A/B Testing

Two versions of a system are exposed to users. User feedback or ratings are compared to determine which version performs better.

4. Golden Dataset / Rubric Creation

Humans can define the correct labels, expected behavior, evaluation criteria, or rubric used later by automated evaluation systems.

5. Human-in-the-Loop

When automated methods cannot confidently evaluate a difficult or ambiguous case, the case is routed to a human for final judgment.

Advantages and disadvantages

7. LLM-as-a-Judge

LLM-as-a-Judge uses one LLM to evaluate the output of another LLM or an LLM-powered application.

In many modern LLM evaluation pipelines, the evaluation method is model-graded or LLM-graded.

The core idea is:

8. LLM-as-a-Judge Case Study: UPSC Mains Evaluation

Business problem

Imagine Campus X UPSC runs mock UPSC Mains exams. Thousands of students may submit subjective answers. Human subject-matter experts would be expensive to hire for every paper.

An LLM-based evaluation platform promises to grade large numbers of answers according to a defined rubric at a fraction of the cost.

Target and task

• Target: The complete answer-evaluation platform.
• Task: Determine whether it evaluates UPSC answers similarly to human experts.
• Success criterion: The platform should evaluate answers similarly to human experts.

Step 1 — Define a rubric

Suppose a question asks:

“Ethical governance is impossible without administrative accountability. Discuss.”

An expert defines dimensions such as:

1. Discusses ethical governance and accountability.
2. Explains the link between them.
3. Provides relevant mechanisms.
4. Uses relevant examples.
5. Provides a balanced conclusion.

This is the rubric, not the dataset.

Step 2 — Build a golden dataset

A human subject-matter expert evaluates around 50–100 student answers using the same rubric.

Example dataset fields:

The human scores become the reference/golden evaluation labels.

Step 3 — Use an LLM as the evaluator

The judge LLM receives:

• The question
• Maximum marks
• The evaluation rubric
• The student's answer
• Instructions for scoring

The judge is instructed to determine whether each rubric dimension is genuinely addressed, allocate marks, avoid rewarding verbosity or keyword stuffing, and provide a concise justification.

Step 4 — Compare human and LLM scores

The goal is for the two columns to be as similar as possible.

9. MAE as the Success Metric

A simple metric for comparing human and LLM scores is Mean Absolute Error (MAE).

For each answer, calculate the absolute difference between the human score and LLM score, then average those differences across all answers.

Interpretation

If MAE = 2.3, the LLM differs from human evaluation by an average of 2.3 marks.

How can the system be improved?

• Use a stronger judge LLM.
• Improve the judge prompt.
• Refine the rubric.
• Add clearer scoring instructions.
• Provide better examples or calibration cases.
• Analyze cases with large human-vs-LLM disagreement.

This creates an iterative evaluation loop: evaluate → identify disagreement → improve judge/rubric/prompt → evaluate again.

10. Reference-Based vs Reference-Free Evaluation

Reference-Based Evaluation

You have a predefined correct/reference answer or expected label for each test case. The generated output is evaluated by comparing it against that reference.

Examples from the lesson:

• RAG retriever evaluation — relevant document IDs are known.
• UPSC LLM judge — human expert scores are known for the golden dataset.

Reference-Free Evaluation

There is no predefined correct answer for each test case. The evaluator judges output quality directly according to a rubric or general criteria.

Example: Human evaluation of chatbot helpfulness. The dataset contains questions, but not a predefined correct score for each generated answer.

11. Offline vs Online Evaluation

The lesson introduces this as the next topic.

Offline Evaluation

Evaluation is performed before or outside production using a fixed evaluation dataset and controlled experiments.

Online Evaluation

Evaluation continues after the system reaches production, using real-world traffic, user behavior, feedback, and production signals.

12. Quick Revision Cheat Sheet

One-line mental model

The three-method mental model