How long is the LLM Evals and Reliability Bootcamp?

12 weeks at about 15 hours a week. The weeks run as six two-week blocks; each block ends in a project you deploy and a gate you must pass before the next block opens.

Is the LLM Evals and Reliability Bootcamp live or self-paced?

Live. It is taught on Zoom by one instructor, with a 30-minute one-to-one every week. Nova, Square 1's AI tutor, grades every exercise and project between sessions against a rubric you can read.

When are the live sessions for the LLM Evals and Reliability Bootcamp?

Live sessions run on weekday evenings from 5:30 PM Sri Lanka and India time, with full-day sessions at weekends. That is 5:00 PM in Pakistan, 5:45 PM in Nepal and 6:00 PM in Bangladesh, so the sessions fit around a working day.

How much does the LLM Evals and Reliability Bootcamp cost?

A$200 for the founding cohort, paid once; the standard price is A$400. It is the same price in every country. Founding students keep their founding price on any second programme.

What will I build in the LLM Evals and Reliability Bootcamp?

6 deployed projects: Golden set, Calibrated judge, Regression in CI, Guardrails, Monitoring and incident and Employer brief. Each is graded against a published rubric and kept on a record an employer can open at /verify.

What skills does the LLM Evals and Reliability Bootcamp teach?

Python, evals, LLM judges, CI, guardrails, monitoring and incident review. It prepares you for roles such as AI quality engineer, ML reliability engineer, AI engineer (evals) and Applied scientist (evaluation).

Does the LLM Evals and Reliability Bootcamp guarantee a job?

No. No placement rate is published because the first cohorts have not graduated. What you leave with is six deployed projects, a recorded defence of your code and a hiring plan worked through with your instructor, including a two-week hiring sprint at the end.

Live bootcampTaught live on Zoom, one instructor

LLM Evals and Reliability Bootcamp

Name: LLM Evals and Reliability Bootcamp
Price: 200.00 AUD
Availability: PreOrder

Make an AI system you can trust: evals, guardrails, monitoring, incident review.

LLM Evals and Reliability Bootcamp is a 12-week live online bootcamp, taught on Zoom by one instructor at about 15 hours a week, with 6 deployed projects and a one-to-one every week. The founding price is A$200, down from A$400. Live sessions run on weekday evenings from 5:30 PM Sri Lanka and India time, with full-day sessions at weekends.

Taught with

PythonClaudeChatGPTGitHub

Not sure it is for you?Take the free Generative AI skill check

By week 12, you will have shipped

Week 1Labelling SheetA labelling tool that shows model outputs one at a time and saves pass or fail labels
Weeks 1–2What good looks likeA labelled golden set and rubric for a real feature, published
Week 3Judge Prompt DraftA judge prompt scored against twenty labelled examples, with an agreement table
Weeks 3–4JudgesA calibrated judge that matches human labels, deployed as a service
Weeks 5–6Regression and CIAn eval pipeline in CI that blocks a bad prompt change
Week 7Prompt Injection FilterA filter that flags injection attempts in user input, tested against a small attack set
Weeks 7–8GuardrailsA guardrail layer with its own evals, deployed in front of a model
Week 9Drift AlertA scheduled job that samples live outputs, scores them and opens a GitHub issue on a drop
Weeks 9–10Production monitoringThe capstone: a monitored LLM feature with alerts and a written incident review
Weeks 11–12Employer brief and hiring sprintThe employer brief shipped, and demo day

weeks, live on Zoom: 12
hours a week: 15
projects to build: 10
minutes 1-1, every week: 30

Who it is for, and what you receive.

Engineers who have shipped an LLM feature and been burned by it, and teams that need someone to own quality.

Before you start

You have shipped an LLM feature and been burned by it.

Skills

PythonevalsLLM judgesCIguardrailsmonitoringincident review

A week, about 15 hours

Live on Zoom: 4h
Set reading and exercises: 6h
The project: 5h

1
Six deployed projects
Each graded by Nova against a published rubric and signed off by the instructor at the gate.
2
The twelve-week plan
Every week has its theme, its topics and what you build, published before you apply and taught live by your instructor.
3
Live on Zoom every week
A 90-minute live code review, a 60-minute squad lab, 60 minutes of office hours, and a 30-minute 1-1 with your instructor.
4
Your hiring plan
Opened in week 1 and reviewed in every 1-1: target roles, the gap map, the proof list, weekly actions, an interview log, the outcome.
5
The record
Every grade and every project with its repository at square1ai.com/u/{handle}, and the recording of your viva, verifiable by any employer.
6
The certificate
A credential ID that resolves at square1ai.com/verify to the real completion.
7
The hiring sprint
Weeks 11 and 12: CV and portfolio from graded work, applications, mock interviews scored against the role, demo day.
8
Nova for twelve weeks
A tutor with every submission of yours in its memory, at 2am as well as in class.

Twelve weeks in six parts.

Each part teaches for a week, then you build and deploy a project, then a gate checks it before the next part opens. The project and its gate are the part's entry on your record.

Live time each week on Zoom: a 90-minute class where the instructor reviews real submissions, a 60-minute squad lab, 60 minutes of office hours, and your own 30-minute 1-1. Nothing is lectured live; the recorded lessons do that.

Key skillsDefining quality for an LLM featureGolden sets and labellingRubrics people agree on
Week 1
What good looks like
- Defining quality
- Rubrics and inter-rater agreement
- Sampling and labelling
- Golden sets
You build: A rubric and a labelling plan
Week 2
Project 1: the golden set
- Labelling 300 cases
- Agreement reports
You build: Labelled golden set, published
Project 1 · the gate at week 2
Golden set
A labelled golden set and rubric for a real feature, published
You hand in
- Rubric
- Labelling guide
- Golden set
- Agreement report
The gate
Two labellers agree above the stated threshold.

What you can do by week 12.

The quality infrastructure every AI team needs and few have: a golden set, a calibrated judge, a CI pipeline, guardrails and monitoring, built once and carried to any team.

1
Define quality for an LLM feature as a rubric people agree on, and build a labelled golden set.
2
Build LLM judges calibrated against human labels and measure their agreement.
3
Run eval suites in CI with change control for prompts and models, and handle flakiness and sampling.
4
Build guardrails for inputs, outputs, prompt injection and PII, and evaluate the guardrails themselves.
5
Monitor an LLM feature in production with online evals, dashboards and alerts.
6
Run an incident review for an AI failure and write it up.

Roles this prepares you for

AI quality engineerML reliability engineerAI engineer (evals)Applied scientist (evaluation)

No placement rate is shown, because there are no graduates to count yet. The roles above are what the projects are built for.

Your record at week 12.

Every exercise and project is graded by Nova against a rubric you can read, and every grade is kept on one page an employer can open and run. This is what the programme writes to it.

Graded, line by line: Nova reads every submission against the brief and the rubric and returns a score, what you did well and what to fix.
Six gates: A block does not open until the previous project passes its gate. You always know where you are and what is next.
A weekly 1-1: Thirty minutes with your instructor, who has already read your code before the call.
One page an employer can run: Every grade, project and the viva recording at /verify. An employer opens it, runs the code and watches you defend it.

Record, LLM Evals and Reliability BootcampExample

Week 2
Golden set
Two labellers agree above a threshold on the set
Graded, gate signed
Week 4
Calibrated judge
Judge agreement with humans is measured and reported
Graded, gate signed
Week 6
Regression in CI
A seeded regression is caught before merge
Graded, gate signed
Week 8
Guardrails
The attack suite is blocked without hurting the golden set
Graded, gate signed

The entries, not the grades: those are yours to earn. The page lives at /verify and an employer needs no account to open it.

8 entries, twelve weeks. Reserving a spot holds the founding price.

Reserve a seat

How we help you find a job.

The last block is not curriculum. It is the hiring sprint, and the proof you built in the ten weeks before it.

The course booklet

The whole plan on a few pages: the parts, the projects, the gates and the price. We email it to you.

Get the booklet

1
Your hiring plan, from week 1
Six parts you and your instructor keep: target roles, the gap map from real postings, the proof to send, weekly actions, an interview log, the outcome. Read before every 1-1.
2
The hiring sprint
Weeks 11 and 12: CV, portfolio, applications, mock interviews, and demo day in front of hiring partners.
3
A record an employer can run
Your deployed projects and the recorded viva on /verify. An employer opens it, runs the code and watches you defend it.
4
The career agent
Paste a real job posting at /career and it maps the role to your graded work and what to do next.
5
The roles directory
Every role we prepare people for, what it pays and what it asks, at /roles.

Questions people ask.

About LLM Evals and Reliability Bootcamp, answered from the plan on this page.

One cohort, one instructor. Hold your spot.

A cohort of 25, one instructor, 12 weeks, six things you ship. Spot holders hear the start date first, and nothing is charged before you confirm.

Six deployed pieces of eval and reliability infrastructure
A calibrated judge and a CI eval pipeline you can carry to any team
A recorded viva
Your record on /verify

A$200~~A$400~~