📚 LLM EVALUATION STUDY HUB

Master LLM Evaluation

Your complete collection of LLM Evaluation study notes. Select any lecture below to open the corresponding HTML study page.

13 Lecture Notes
01

Master LLM Evaluations.

Why LLM evals exist, how they differ from software testing, and the core definition every engineer should know.

Open →
02

Introduction to LLM Evaluations: Model Evals vs Application Evals

The core split every eval strategy is built on — evaluating the raw model vs. evaluating the full application around it.

Open →
03

How to Evaluate LLM Applications: The Complete Workflow

A step-by-step, repeatable workflow — from defining success criteria to deployment and monitoring.

Open →
04

Why Your AI Application Needs Multiple Eval Pipelines?

One application, many evals — why component-level and system-level testing both matter.

Open →
05

LLM Eval Methods | LLM-as-a-Judge | Reference Based Evals Vs Reference Free Evals

Choosing between automated scoring, human review, and LLM-as-judge for your evaluation pipeline.

Open →
06

Offline Evals Vs Online Evals

When to test before deployment vs. continuously in production, and why you need both.

Open →
07

LLM Model Evals & Capabilities

The eight capability areas frontier labs test — reasoning, knowledge, coding, tool use, and more.

Open →
08

What is LLM Benchmarking | Benchmark Saturation vs. Contamination

Why benchmarks stop being useful over time, and how to spot when a score can't be trusted.

Open →
09

What are LLM Benchmarks | The Evolution of AI Knowledge Benchmarks

A tour of how knowledge benchmarks have evolved as models have gotten more capable.

Open →
10

How to Use LLM Leaderboards

Reading leaderboards critically to pick the right model for your use case.

Open →
11

Selecting the Right LLM for Your AI App: Running Custom Model Evals

Building your own targeted model evals when public benchmarks don't match your real use case.

Open →
12

How to Answer "How Do You Evaluate Your RAG App?" in GenAI Interviews

A structured, interview-ready framework for explaining RAG evaluation end to end.

Open →
13

How to Test RAG Retrievers (Hands-On)

A practical, hands-on walkthrough of testing and scoring a retriever's performance.

Open →

cheetsheet1

cheetsheet2

cheetsheet3

cheetsheet4

cheetsheet5

cheetsheet6

cheetsheet7

cheetsheet8

cheetsheet9

cheetsheet10

cheetsheet11

cheetsheet12

cheetsheet13

overall cheetsheet

llm evaluation part2