Courses · Level 4 Production
Evals, or It Didn’t Happen
You cannot improve what you refuse to measure.
You cannot improve what you refuse to measure.
Build an eval set on the page, score two versions of a system against it, and catch a regression yourself — the nasty kind, where the new version scores better on average while being completely broken on one category. Then the honest treatment of LLM-as-judge: when it works, how it lies, and how to check it.
Press play and a teacher talks you through the whole build, with a read-along transcript that highlights as it speaks.
Working, offline interactives — not screenshots. When the narration says "try it", the thing is right there.
No account, no app, no subscription. It runs from a single HTML file and will still run in ten years.
Every number is computed live on the page from inputs you can edit. Nothing is asserted that cannot be checked.
Written for teams shipping AI features they have to stand behind.
Build an eval set with golden, edge, and adversarial cases
Pick a scoring method that matches the task
Catch a category regression an average would have hidden
Know when an LLM judge can be trusted, and how to verify it
13 chapters, the first four free
The free preview is not a teaser: those chapters are complete, and the paid chapters are physically absent from the preview file rather than hidden behind a script.
What this hour is, how to use the page, and the interactive demo you will keep coming back to.
Part of the free preview — the full narration, page content and exercises, nothing held back.
Part of the free preview — the full narration, page content and exercises, nothing held back.
Part of the free preview — the full narration, page content and exercises, nothing held back.
Narrated in full, with the on-page build and exercises for this idea.
Narrated in full, with the on-page build and exercises for this idea.
Narrated in full, with the on-page build and exercises for this idea.
Narrated in full, with the on-page build and exercises for this idea.
Narrated in full, with the on-page build and exercises for this idea.
Narrated in full, with the on-page build and exercises for this idea.
Narrated in full, with the on-page build and exercises for this idea.
Narrated in full, with the on-page build and exercises for this idea.
Narrated in full, with the on-page build and exercises for this idea.