1. What Are LLM Evaluations?
LLM evaluations are systematic, repeatable tests used to judge an LLM or an LLM-powered system against clear evaluation criteria.
LLM evaluations are not simply a collection of random questions asked to a model. They are structured testing systems designed to determine whether a model or LLM-powered application behaves as expected.
The definition contains three critical characteristics:
1. Systematic
Testing follows a structured methodology rather than relying on a few manually selected prompts.
2. Repeatable
The same evaluation can be executed again against different versions of the system.
3. Clear Criteria
The system is evaluated against explicitly defined criteria rather than personal intuition.
2. The Three Core Properties of LLM Evals
Systematic Testing
Systematic testing means that evaluation is performed using a properly prepared dataset rather than casually asking a few questions.
For example, imagine building a chatbot for an educational platform. Instead of manually asking five questions, you could collect a representative sample of real user conversations and convert them into an evaluation dataset.
Goal: Cover normal cases, difficult cases, edge cases, and real-world user behavior as comprehensively as possible.
Repeatable Testing
Repeatability means that the same evaluation dataset should continue to work even when the system changes.
For example, you might change:
- Prompt
- LLM
- Retriever
- Chunking strategy
- Embedding model
- Application logic
Even after these changes, you should be able to run the same evaluation suite and compare the results.
This allows developers to determine whether a new version actually improved the system or introduced regressions.
Clear Evaluation Criteria
The third requirement is defining what "good" actually means for your application.
For example, a chatbot might need to satisfy the following criteria:
- Answer must be correct.
- Explanation should be simple and understandable.
- Answer should use the approved knowledge base.
- Response should be grounded in retrieved information.
- Response should be safe.
- Response should not contain abusive language.
- Response should maintain the expected tone.
Key idea: Without explicit criteria, you are essentially doing informal testing. With explicit criteria, you are performing structured evaluation.
3. LLM Evals Are Not Just Metrics
A common misunderstanding comes from traditional machine learning.
In classical ML, evaluation often means calculating metrics such as:
- Accuracy
- Precision
- Recall
- F1 Score
- ROC-AUC
This can create the misconception that an LLM evaluation is simply a set of metrics.
An LLM eval is the complete testing setup, not merely a metric.
The complete evaluation setup can include:
- What component is being tested?
- What dataset is being used?
- What criteria are being evaluated?
- What evaluation method is being used?
- Which evaluation tools are being used?
- When is the evaluation performed?
- Is evaluation offline or in production?
- How are the results interpreted?
For example, if you are evaluating the retriever inside a RAG system, the retriever, evaluation dataset, retrieval metrics, evaluation framework, and execution process can all be considered parts of the evaluation setup.
4. What Is the Purpose of an LLM Eval?
The goal of an LLM evaluation is not simply to produce a number.
The goal is to answer practical engineering and product questions.
Can We Use This Model?
Is this model capable enough for the task we want to solve?
Is It Ready to Ship?
Is the application reliable enough to be deployed to real users?
Did the Prompt Improve?
Did Prompt Version 2 actually perform better than Prompt Version 1?
Is the RAG Grounded?
Are responses actually supported by the retrieved context?
Is the Agent Correct?
Is the agent completing the intended task correctly?
Is the System Safe?
Can real users safely interact with the system?
Is Latency Acceptable?
Does the application respond quickly enough for the intended user experience?
5. Two Major Types of LLM Evals
LLM evaluations can broadly be divided into two categories:
| Type | What Is Evaluated? | Main Purpose |
|---|---|---|
| Model Evals | The LLM itself | Measure the capabilities and performance of the underlying model. |
| Application Evals | An LLM-powered application or its components | Determine whether the complete product or individual components work correctly. |
Important terminology note: "Model Evals" and "Application Evals" are useful categories for understanding the topic, but they are not necessarily official industry terms. In practice, both are often discussed broadly as "LLM evaluations," with the context determining which type is meant.
6. Model Evaluations
Model evaluations focus directly on the underlying LLM.
Evaluating the capabilities of an LLM itself using standardized tests and benchmarks.
Whenever a new LLM is released, its creators typically evaluate it across different benchmarks and publish the results.
This helps the industry understand how capable the model is compared with other models.
These evaluations help developers and organizations understand which models might be suitable for their particular applications.
7. Eight Major LLM Capability Categories
Modern LLMs are commonly evaluated across several major capability areas.
Reasoning
Can the model reason through a problem step by step and arrive at an appropriate solution?
Knowledge
Does the model possess sufficient general and world knowledge up to its knowledge cutoff?
Basic Mathematics
Can the model correctly solve mathematical problems?
Coding
Can the model understand programming problems, generate code, and solve coding tasks?
Instruction Following
Can the model follow multiple instructions accurately and consistently?
Long-Context Handling
Can the model retrieve and reason over relevant information from very large contexts?
Multimodal Understanding
Can the model understand and work with different modalities such as text, images, audio, and other forms of input or output?
Tool Use
Can the model correctly use external tools, APIs, functions, or other capabilities?
8. Benchmarks Used for Model Evaluation
Model capabilities are evaluated using standardized benchmarks.
| Capability / Area | Example Benchmark | Purpose |
|---|---|---|
| Knowledge & Reasoning | MMLU | Tests knowledge and reasoning across multiple subjects. |
| Mathematics | GSM8K | Tests mathematical problem-solving, especially grade-school-level problems. |
| Coding | SWE-bench | Evaluates models on real-world software engineering tasks. |
| Human Evaluation | HumanEval | Evaluates code-generation capabilities. |
| Instruction Following | IFEval | Tests whether models follow explicit instructions and constraints. |
| Long Context | Needle in a Haystack | Tests whether a model can retrieve specific information from a long context. |
| Multimodal Understanding | TRIPLE-MU | Used to evaluate multimodal capabilities. |
Important: These benchmarks are examples. The benchmark landscape is much larger, and different benchmarks measure different capabilities and behaviors.
9. How Important Are Model Evals for an AI Engineer?
AI engineers should understand model evaluations, but they generally will not be responsible for creating the major industry-wide benchmarks used to evaluate frontier models.
Frontier AI labs typically perform large-scale model evaluations when they release new models.
However, AI engineers should understand:
- What model evaluations are.
- What benchmarks are.
- What different benchmarks measure.
- How to interpret benchmark results.
- How benchmark results can influence model selection.
Practical benefit: When starting a project, understanding benchmark results helps you make better decisions about whether to use a proprietary model, another provider's model, or an open-source model.
10. Application Evaluations
Application evaluations are the most important part of this learning path for an AI engineer.
Application evaluations assess the behavior and performance of an LLM-powered application, either at the level of the complete system or at the level of specific components.
A common beginner misconception is:
"The LLM is the brain, so if the LLM is good, the application must also be good."
This is incorrect.
An LLM-powered application usually contains many additional components.
User Interface
The interface through which users interact with the application.
System Prompt
Instructions controlling the model's behavior.
Tools & APIs
External functions, APIs, databases, and services used by the application.
Orchestration
Logic controlling workflows, branching, parallel execution, and agent behavior.
Guardrails
Safety and behavioral controls around the application.
Output Parsers
Components responsible for converting model output into structured formats.
Memory & Context
Mechanisms that maintain relevant information across interactions.
RAG Components
Retrieval systems, embedding models, vector databases, rerankers, and related components.
Monitoring
Production monitoring and observability systems.
Feedback Loop
Mechanisms for collecting user feedback and improving the system.
11. Smartphone Analogy
A useful way to understand application evaluations is through a smartphone analogy.
A smartphone may contain a powerful processor such as a Snapdragon or MediaTek chip.
The processor itself can be benchmarked to determine how powerful it is.
But does having a powerful processor automatically make the smartphone a great product?
Obviously not.
A smartphone also depends on:
- Camera system
- Operating system
- Battery
- Display
- Audio system
- Graphics capabilities
- Software optimization
- Connectivity
A powerful processor tells us something about the processor. It does not guarantee that the complete smartphone is a good product.
The same principle applies to LLM applications.
Frontier labs evaluate the underlying models. AI engineers are responsible for evaluating the application built around those models.
12. Application Evals: System-Level vs Component-Level
Application evaluations can happen at two levels:
| Level | What Is Evaluated? | Example |
|---|---|---|
| System Level | The complete LLM-powered application. | Is the final RAG chatbot response correct, grounded, safe, fast, and useful? |
| Component Level | Individual components within the application. | Is the retriever working correctly? Is the embedding model effective? Is the reranker performing correctly? |
For example, in a RAG chatbot you might separately evaluate:
- Retriever quality
- Embedding quality
- Reranker performance
- Context relevance
- Answer correctness
- Groundedness
- Latency
- Cost
- Safety
13. Model Eval vs Application Eval
| Question | Model Eval | Application Eval |
|---|---|---|
| What is being evaluated? | The underlying LLM. | The complete application or one of its components. |
| Main question | "Can the model do this?" | "Will our product work correctly?" |
| Typical owner | Frontier model labs. | AI/application engineering teams. |
| Typical tools | Standardized benchmarks and leaderboards. | Custom datasets, rubrics, evaluators, testing frameworks, and application-specific metrics. |
| Example | Can the LLM solve mathematical problems? | Does our RAG chatbot provide grounded and correct answers? |
14. Questions Application Evals Should Answer
For an AI engineer, application evaluation should help answer questions such as:
Correctness
Did the application provide the correct answer?
Groundedness
Is the answer supported by the retrieved context?
Completeness
Did the system provide all important information?
Clarity
Is the answer understandable to the intended user?
Safety
Is the application safe for real-world users?
Latency
Does the application respond quickly enough?
Cost
Is the application economically viable at scale?
Task Completion
Can the agent or application complete its intended task correctly?
15. Why Application Evals Matter More to AI Engineers
The job of an AI engineer is to build LLM-powered applications.
Therefore, understanding whether the complete application actually works is a core engineering responsibility.
A model can have excellent benchmark scores while the application built around it can still fail because of:
- Bad prompts
- Poor retrieval
- Incorrect chunking
- Bad embeddings
- Incorrect orchestration
- Tool failures
- Unsafe outputs
- Hallucinations
- High latency
- High operational cost
A good model does not automatically create a good application.
16. Complete Lecture Summary
This lecture answered the "WHAT" behind LLM evaluations.
Why?
We need LLM evaluations because LLM-powered applications are probabilistic and complex, making traditional software testing insufficient.
What?
LLM evaluations are systematic, repeatable tests performed against clearly defined criteria.
Two Types
Model evaluations measure the capabilities of the underlying LLM, while application evaluations measure the behavior of the application built around it.
How?
The next stage is learning how application evaluations are actually designed, implemented, and executed.
17. Key Takeaways
- LLM evals are not just metrics. They represent the complete testing setup.
- Good evaluations are systematic. They use structured datasets rather than random manual testing.
- Good evaluations are repeatable. The same evaluation should work across different versions of the system.
- Evaluation requires clear criteria. You must define what "good" means before testing.
- There are two useful categories: model evaluations and application evaluations.
- Model evaluations measure model capabilities.
- Application evaluations measure the product.
- Model evaluation commonly uses benchmarks.
- Modern LLMs are evaluated across multiple capabilities including reasoning, knowledge, mathematics, coding, instruction following, long context, multimodality, and tool use.
- AI engineers should understand model benchmarks even if they rarely create frontier-model evaluations themselves.
- Application evaluation is the core practical focus for AI engineers.
- An LLM is only one component of an LLM-powered application.
- Application evaluation can happen at both system and component levels.
- The key question changes: Model eval asks "Can the model do this?" Application eval asks "Will our product work correctly?"
18. What Comes Next?
How are LLM evaluations actually performed?
The next part moves from the "WHAT" of LLM evaluations toward the "HOW", with the main focus on Application Evaluations.