What Week 4 builds
Is my agent team actually good, and how would I know?
- Source
- Internal's lesson, src/week4/internal/lessons/anw-w4-m0.md (Internal 60d90336, 2026-10-19 19:39), read by md_week4.py.
- Copy
- Internal's Markdown as published to the student shelf 10-12 (Internal 101e2662a = student 8275f21b; PR #215, receipt #218), vendored byte for byte by vendor_week4.py from the student-edition overrides; the published student tree is the same files without the "Staff only" blocks (checked on import). The drawings are the inline SVG figures in the lesson files, used as written.
- Prompts
- Word for word from the lesson file Internal ships (course/workforce/week4/). Not run end to end from Learn.
- Checked in the app
- Not checked in the app from Learn.
- Open to students
- Yes.
Staff only: source, checks, open points. Not student-facing.
Source. Week 4 spine v2 (aibuild-lab/course-design, courses/agent-native-workforce/intake/week4-lessons/SPINE-week4-v2-2026-10-08.md at dd055223), Module 0, in the shape of the Week 3 welcome. Tyler, 8 October: the whole week ships Monday 12 October, Modules 0 to 6 together. Read only: no prompt file, nothing is saved. No dates on the page (spine).
Times are the sums of the lessons' own part minutes: Module 1 85 min (l01 35, l02 50; l02 Part 5 raised from 10 to 15 min by Sara, 10 October); Module 2 60 min (l03 30, l04 30); Module 3 85 min plus agent time (l05 40, l06 45); Module 4 40 min (l07); Module 5 90 min (l08 30, l09 30, l10 30); Module 6 100 min (l11 40, l12 30, l13 30; l13, What's next, after the course, added 13 October). Week total about 7 hours 50 min plus agent time, against the spine's 6 h 35. l02 grew by the per-rule grading and seal (Codex review item 5), l11 by making every golden case pass and the two measures (items 11 to 13). Sara to rule on the overrun (spine open item 4).
Names students see follow spine open item 3, ruled by Sara 11 October (the correct terms, not replacements): "golden examples", "golden set" and "golden case", "regression test"; still pending: "Evy's notebook" (the spine's "Evy's notes on how you grade"), "watch list", "proof page", "course mode" and "everyday mode".
| Figure ID | Lesson | What it shows | Caption | Alt text |
|---|---|---|---|---|
w4-m0-path | anw-w4-m0 How it works | Six boxes left to right with arrows: grade it yourself, Evy grades it too, fix one problem, keep the good, check every day, prove it | One week, six moves, each one using what the one before it made. | Week 4 in six steps: you grade your team's work, Evy grades the same work, you fix one problem, you keep golden examples and build your golden set, your organization checks itself every day, and you build a page that proves it. |
Last week you built teams and watched them work. This week you find out how good they are, and you set your organization up to keep checking.
You grade your agent team's work first, in your own words. Then the Evaluation Analyst (Evy) grades the same work without seeing your grades, and you compare. Where you disagree, either your rule was unclear or Evy was wrong, and you fix whichever it was. Evy keeps notes on how you grade, so over time you check less and Evy checks more.
Then you fix one problem with the Operator (Gigawatt) and prove the fix worked without breaking anything else. By the end of the week, your organization checks its own work every day and tells you in plain words how each team did.
One team with three cases is enough. If you only finished your email team last week, do the whole week on that team. More teams means more to grade, not a different week.
In this course, your team always means a team of agents, like your email team, never the people you work with.
A rule sheet for your team, an evaluator that grades the way you do, one problem fixed and proven, golden examples, a golden set with regression tests, a daily check across every team, a dashboard to see it all, and a one-page proof of what your organization does now.
A team that looks fine once might be lucky. Grading its work against your own rules, every day, is how you know it's good, and how you catch it the day it slips.
Seven modules, in order. Each one uses what the one before it made.
You grade first, in your own words. You decide what your rules say, which problem to fix, whether a fix stays, and which work counts as great. Nothing changes on your team without you: you approve the plan for a fix, and you decide whether it stays. Your agents do everything in between.
What you get
Your golden set is the saved cases your team must keep getting right, rerun after every change. Your golden examples are your best work, kept so your team can see what good looks like.
How it works
One week, six moves, each one using what the one before it made.
| Module | What you do |
|---|---|
| 0 · Welcome | This page. |
| 1 · Grade it yourself first | Give each case a pass or fail with your reason, then turn what you found into the rules for your team. Your grades are sealed before Evy sees anything. |
| 2 · Evy grades the same work | Evy grades the same cases against your rules. You compare, and teach Evy where you disagree. |
| 3 · Fix one problem and prove it | Pick the biggest problem, approve Gigawatt's fix, rerun the golden set, and keep the fix or undo it. |
| 4 · Keep the good, save the bad | Save your best work as golden examples, and turn each mistake into a regression test that must keep passing. |
| 5 · Let your organization check itself | Turn on Evy's daily check, open your dashboard, and make sure everything your agents do is recorded. |
| 6 · Prove it, and take it to work | Build one page that shows what your organization does now, then move it from class to everyday work. |
We show you first. Then you do it.
In every lesson, we show you each step first on the Hattie B's practice inbox from last week. You watch an example student do it. They grade the three practice emails you ran last week: the kids plate question, the catering quote and the fake bank email. They also ran two more emails from the same pack: the cold-food complaint, and Marcus asking about "Saturday", which turns out to be Valentine's Day.
Why Hattie B's, when you built your own team? Everyone in the course ran the Hattie B's emails, so it's one example we can all look at together. We can't see your team. So we show each step on Hattie B's, and you do it on yours.
Then you do the same step yourself, on your own team and the cases it ran last week. If you built a team with the Architect (Archie) and the Operator (Gigawatt), that's the team you grade.
If you only finished your email team, your cases are the practice emails your team answered last week. They're still your team's work, and you grade them yourself.
The Course Companion (The Professor) guides every lesson.
Who to ask. This week you work with three agents. Each has one kind of job:
| You want | Ask | Think of it as |
|---|---|---|
| Work done | Your Chief of Staff | Like a real chief of staff, it runs the day-to-day: it gets your team to do the work, like rerunning the tests. |
| Work graded | Evy | Your auditor. It checks the work and never does it, so its grades stay honest. |
| Your team changed | Gigawatt | Your builder. It changes how your team works, but only the way you approved. |
What to have ready
- What Week 3 handed over. The cases your team ran, with your notes on what you expected before each run. If you finished all of Week 3, "Get ready for Week 4" already checked your teams are on the record. If you stopped after your email team, that's fine: the first lesson this week has a short catch-up that puts it on the record.
- This week's course files. Your workbench has the folder
course/workforce/week4/. If it's missing, ask The Professor what to do before you start. - One team and three cases, at least. Which three depends on how far you got last week:
- A team you designed: cases 1, 3 and 5 from your design. That's the ordinary job, the one with missing information, and the one the team must refuse.
- Your email team: the three Hattie B's practice emails you ran last week. Each one is one of those same three kinds of case:
- the kids plate question is the ordinary job
- the catering quote is the one with missing information
- the fake bank email is the one the team must refuse
Set aside the time
| Module | About |
|---|---|
| 1 · Grade it yourself first | 1 hour 25 min |
| 2 · Evy grades the same work | 1 hour |
| 3 · Fix one problem and prove it | 1 hour 25 min, plus agent time for the rerun |
| 4 · Keep the good, save the bad | 40 min |
| 5 · Let your organization check itself | 1 hour 30 min |
| 6 · Prove it, and take it to work | 1 hour 40 min |
Your agents also need time to work. Start a run, then do something else while it finishes.
Next: Read your team's work and give each case a pass or fail.
Words you'll meet this week
- pass or fail, at two levels: the job level and the agent level
- rule sheet, and sealed grades
- Evy's notebook, and how often you and Evy agree
- watch list, and change record
- golden examples, golden set, and regression tests
- Evy's daily check, the dashboard, and Langfuse
- proof page, course mode and everyday mode
A short glossary
These are the words you'll meet outside this course, too.
- Golden set: the saved cases your team must keep getting right, rerun after every change.
- Golden case: one case in your golden set, with the answer it should give.
- Golden example: a piece of your team's best work, kept to show what good looks like.
- Regression test: a golden case you add after a mistake, so the mistake can't come back unnoticed.
- Rubric: a list of yes-or-no rules work is graded against. In this course it's your rule sheet.
- Eval: short for evaluation, grading work against a rubric. In this course it's grading, and the Evals tab on your dashboard.
- LLM-as-judge: an AI model that grades another AI's work. In this course that's the Evaluation Analyst (Evy).