Save your golden examples, and turn mistakes into regression tests
How does my agent team remember what good looks like, and how do I stop an old mistake from coming back?
- Source
- Internal's lesson, src/week4/internal/lessons/anw-w4-l07.md (Internal 60d90336, 2026-10-19 19:39), read by md_week4.py.
- Copy
- Internal's Markdown as published to the student shelf 10-12 (Internal 101e2662a = student 8275f21b; PR #215, receipt #218), vendored byte for byte by vendor_week4.py from the student-edition overrides; the published student tree is the same files without the "Staff only" blocks (checked on import). The drawings are the inline SVG figures in the lesson files, used as written.
- Prompts
- Word for word from the lesson file Internal ships (course/workforce/week4/). Not run end to end from Learn.
- Checked in the app
- Not checked in the app from Learn.
- Open to students
- Yes.
- Other view
- one part at a time (one part at a time is the lesson; one scroll is kept to compare)
Staff review only: full controlling prompt. Students paste only the short request below.
Use the native aibl-the-professor agent to guide the Week 4 lesson Save your golden examples, and turn mistakes into regression tests. Read course/workforce/week4/lessons/anw-w4-l07.md completely, its details page course/workforce/week4/reference/anw-w4-l07-details.md when asked for more, and the native aibl-course-guide and aibl-lesson-map methods for this app. You teach and read; you never choose a golden example, write a note, save a test, or review a flagged one for the student. The student talks to the Chief of Staff in its own thread. Do not call, spawn or route to any agent. Do not edit any file.
Use these words exactly. Say golden set, golden case, golden example and regression test; never saved tests or examples of great. Each team has two golden collections (the build calls them shelves; never say shelf or card to the student: say golden examples, golden set, golden case, and note), kept with the aibl-golden-shelf method in the team's folder under work/course/staff/teams/: great/ holds golden examples, which the team's agents read before similar work as examples, never as templates, and which the Evaluation Analyst (Evy) grades against; tests/ holds the golden set, the team's design cases plus every failure turned into a regression test, which must keep passing after every change. Every card says where it came from. Made by a person: who, from where (the tool and a reference), and when. Made by the team: when, which job, which app and app version, which model each agent used, and which version of each agent. Every card also lists the facts and the agents it depends on, why it is great or what it must never do again, who approved it in their own words, and its status: active, needs review, or retired. When a fact a card depends on changes, or the change record logs a change to an agent, a source or the design, a fixed script flips every card that depends on it to needs review; a card not reviewed in 90 days is flagged too. A flagged card is left out of grading and out of the team's reading until it is reviewed: Evy proposes keep, edit or retire, the student asks Evy directly for the proposals (Evy never takes work from the Chief), confirms each decision in their own words, and the Chief saves it with the aibl-golden-shelf method. An example with no source behind it is never shelved. From now on, every new failure is offered to the student as a new regression test, and saved only when the student approves it. Name agents job title first, nickname in brackets on first use; Evy is "it". Say "your work", never "business"; refer to other lessons by title, never by number.
The Hattie B's practice inbox is the worked example on the page. Never reveal or paraphrase any answer key for the practice emails; use only what the lesson page shows.
Ask open questions only. Never pick which work is great or which mistake to save.
Part 1. If asked, explain golden examples and the golden set and what each is for, from the page.
Part 2. If asked, read a note the student shows, read only, and check it says where it came from with every field the page lists. Say plainly which field is missing.
Part 3. If asked, read the list of this week's failures the student shows, read only, and help them see which are already golden cases.
Part 4. If asked, read the facts an example or test depends on and explain what would flag it for review.
Otherwise, if asked, explain the step and what the student should see.
Staff only: status, sources, what the build supports. Not student-facing.
Status: first draft, 8 October 2026, lane "lessons" (dispatch 51bbe9da), from spine v2 Module 4 and DESIGN-week4-2026-10-08.md section 7 (shelves work/course/staff/teams/<id>/great/ and tests/, facts-index.yaml, reviews/; card fields; human and team origins, Tyler 8 October; upkeep by a fixed script on a fact change, a change-record entry, or 90 days; flagged cards left out; Evy proposes, the owner confirms, Sallee files; "a golden with no source is folklore").
Bound to the merged lanes where they have landed; plain words elsewhere.
- Bound to PR #199:
aibl-golden-shelfwithgolden_flags.py(runtime) as the only writer (add,promote,scan,review,active,check). The Chief of Staff'smay_writein that PR coverswork/course/staff/teams/<id>/great/,tests/,facts-index.yamlandreviews/, only throughgolden_flags.py, so the page has the Chief save cards on the student's exact words. A golden case'sgolden:line in the team record is the Charter Steward's (skill text). - Real past examples on the shelf: Week 3's design step can now pull them (PR #196, "real examples from the owner's tools, on one 'can I look' yes"). Seeding the shelf from them is the Week 3 thread's job. The page says "if", so it stays true for a student whose design found none.
- The Hattie B's cards below are a made-up worked example. The stale-fact example (dropping the Shut the Cluck Up heat level) is the design's own; the level is in the pack's
menu.md.
| Figure ID | Lesson | What it shows | Caption | Alt text |
|---|---|---|---|---|
w4-l07-two-shelves | anw-w4-l07 Part 1 | Two rows of named items, drawn by l07_figs_svg.py. Top, GOLDEN EXAMPLES: "Catering reply", "Group booking", "Your next one", with arrows out to "Your team reads them before similar work" and "Evy grades against them". Bottom, GOLDEN SET: "Kids plate", "Fake bank", "Marcus", an arrow in from "A regression test, when you approve it" and an arrow out to "Must keep passing after every change" | Golden examples show your team what good looks like; your golden set stops old mistakes coming back. | See the figure's title element. |
Today you build two collections for your agent team. The first is your golden examples: best-in-class results that show what good looks like. Your team reads them before similar work, so it knows what good looks like, and the Evaluation Analyst (Evy) grades against them. The second is your golden set: your Week 3 test cases, plus a regression test for any mistake you choose to keep out. They must keep passing after every change, so a mistake you fixed once stays fixed.
Every example and test says where it came from: who made it, when, and with what. When a fact behind it changes, it's set aside until you check it.
Golden examples and a golden set for your team.
Your agent team learns from good examples, and a mistake you fixed once stays fixed.
The Course Companion (The Professor) guides you in its own thread. Your Chief of Staff keeps your team's records, so it shows you what's there and saves each new one when you approve it.
You choose what counts as great, and which mistakes become regression tests.
Before you start
- Your fix is kept or undone. You did it in "Make the fix, rerun the golden set, and keep or undo it".
- Nothing to open. Your Chief reads your team's work, your golden examples and your golden set.
- In Codex, start your first message to an agent in a thread with "Ask the <agent> to do this, and show me its reply word for word:". Start every message after that with "Tell the <agent> this, word for word, and show me its reply word for word:".
Who to ask. This lesson uses two agents. Each has one kind of job:
| You want | Ask | Think of it as |
|---|---|---|
| Your golden examples or golden set shown, or a new one saved | Your Chief of Staff | Like a real chief of staff, it runs the day-to-day: it keeps your team's records and saves each new one when you approve it. |
| Advice on an example or test that needs review | Evy | Your auditor. It proposes whether to keep, change or remove it, and never saves anything itself. |
Start or resume this lesson
Open your prepared workbench and call The Professor by its role name, aibl-the-professor. Paste only this.
Use the native aibl-the-professor agent to start the Week 4 lesson Save your golden examples, and turn mistakes into regression tests. Read course/workforce/week4/prompts/anw-w4-l07.txt and course/workforce/week4/lessons/anw-w4-l07.md. Ask me one question at a time and wait. I will talk to my Chief of Staff in its own thread.
The full instructions your agent follows for this lesson are in course/workforce/week4/prompts/anw-w4-l07.txt.
First you look at what your team already has, so you know what's there before you add anything.
Golden means trusted: something you hold everything else up against. Your agent team keeps two golden collections, because they do two different jobs:
- Golden examples are best-in-class results: outputs that show exactly what good looks like. Your team uses them as its reference before it does similar work.
- Your golden set is your team's standard exam: the same test jobs every time, each with a known right answer. You started it in Week 3 (for the email team, jobs like the kids plate question and the fake bank email). Every time you change your team, it sits the same exam again. Because the questions never change, if a job that used to pass now fails, you know your change caused it. Each test job is a golden case.
A regression test is a question you add to the exam after your team gets something wrong. "Regression" means sliding backwards: the test makes sure a mistake you've fixed doesn't creep back in after a later change. Say your team once promised a refund it shouldn't have. You add that job to your golden set, and from then on every change is checked against it.
These are the words you'll hear for this outside the course too.
Each golden example and golden case also has a short note: where it came from, who approved it, and why it's there.
Golden examples show your team what good looks like; your golden set stops old mistakes coming back.
Your Week 3 test cases already rerun after every change, as you saw in the last lesson. Most of them don't have a note yet saying what each one must never do. You'll add those today.
Do this now: look at what you have
Your Chief keeps your team's records, so ask it. In a new thread with your Chief of Staff, say this, with your team's name:
Show me my <team name> team's golden examples and golden set, one line each, with where each came from.
What you should see
One line for each golden example and golden case, with where it came from. Most students see their golden cases from Week 3, mostly without notes, and few or no golden examples yet. That's normal: you add them in the next two parts.
Now you choose work you'd happily send again, and say why. Each one becomes a golden example your team learns from.
Every golden example records where it came from
Each golden example has a note saying where it came from. Your Chief fills it in; you just check it's there.
- Made by your agent team: which job, when, and how your agents were set up at the time.
- Made by you (say, an email you sent): where it's kept, and when you wrote it.
Why it matters: your agent team will change. The note tells you whether an example came from the agents as they are now, or as they used to be. And if nobody can say where an example came from, it doesn't go in.
We show you: the Hattie B's practice inbox
The student chose two golden examples:
| Example | Made by | Why it's great, in the student's words |
|---|---|---|
| The catering quote reply | The agent team: the catering practice job, this week | "It answers every question the evidence covers, and leaves the price and the date for me, saying I'll confirm within a day instead of guessing. It uses our greeting and sign-off, and it passes all four of my job-level rules." |
| A reply about a large group booking | The student: from their sent mail, last spring | "It thanks them, says what we can do for a group that size, and says exactly what happens next and by when, in under a hundred words. That's the tone I want in every booking reply." |
A good reason names exactly what makes the work great, so your agents and Evy know what to look for. "It's polite" isn't enough; say what it does.
This is the full note the Chief showed for the catering reply, before the student approved it:
Golden example: the catering quote reply
Covers: the whole job (job level)
Made by: your agent team, the catering practice job, this week, in Claude Code, with how each agent was set up at the time
Why it's great: "It answers every question the evidence covers, and leaves the price and the date for me, saying I'll confirm within a day instead of guessing. It uses our greeting and sign-off, and it passes all four of my job-level rules."
Depends on: the catering section of the policies; the brand voice guide; the Support Lead, the researcher, the writer and Kansa
Status: active once you approve it
Approved by: waiting for your words
Your team reads examples like these as examples, not templates: it learns what good looks like, and still writes each reply fresh.
Do this now: choose your examples
Pick one to three pieces of your agent team's work from this week that you'd send or use again as they are. If nothing is that good yet, that's fine: save none, and add one when it comes. Use each case's name as your Chief listed it. In the same Chief thread, for each one, say:
Add <the case or job> to my team's golden examples. Why it's great: <exactly what makes it great, in as much detail as you need; name the rules it passes if you can>. Show me its full note and save it when I approve it.
Check the note has every line, including where it came from and your full reason. Then approve it in your own words; those words go on the note.
You can also add something you made yourself, like an email you sent, or one of the past examples the Architect (Archie) found when it designed your team. Tell your Chief which one, and why it's great.
What you should see
A full note for each example, like the one above: what it covers, where it came from, your reason, what it depends on, and its status. It's saved only after you approve it.
Now you make sure this week's mistakes can't come back. Here's how, on one example.
We show you: the Hattie B's practice inbox
On the cold-food complaint, the agent team promised a refund it wasn't allowed to give. To stop that happening again, the student:
- Kept that email in the golden set. Every time the agent team changes, it answers that email again, so the student will see straight away if the mistake comes back. It was already there, from Week 3.
- Wrote what it must never do, in their own words: "Never promise a refund. Refunds are my call." That tells anyone looking what the test is checking.
The student did the same for Marcus: "Never sign off in a different voice. Use the practice voice's sign-off."
Do the same for each mistake you want kept out. Most are already in your golden set, so usually you just write the line.
Do this now: save your mistakes
In the same Chief thread, say:
List every case that failed this week on my <team name> team: the ones I graded as a fail, the ones Evy graded as a fail, and any that failed in the rerun. For each one, say whether it's already a golden case, and whether it has a note.
What you should see
A list of every failure, each marked golden case or not, and note or no note.
For each one without a note, or not yet a golden case, decide whether you want it kept out for good. If you do, say:
Make <the failure> a golden case with a note. It must never: <your words>. Show me the note and save it when I approve it.
What you should see
A note for each one, with what it must never do, in your words. It's saved only after you approve it.
From now on
Whenever your team fails at something new, your Chief offers to save it as a regression test and add it to the golden set that reruns after every change. It's saved only if you approve it.
Your work changes: prices, menus, policies, and your agents themselves. When it does, some golden examples and golden cases stop being right. This part shows how you keep them current.
How you know one is out of date
Each note lists the facts and agents it depends on. It's flagged for review when:
- a fact or an agent it depends on changes
- nobody has looked at it in 90 days
For the Hattie B's catering reply, the note shows it like this:
Golden example: the catering quote reply
Status: active. Last checked: today.
Depends on these facts:
- catering lead times (the catering section of the policies)
- the greeting and sign-off (the brand voice guide)
Depends on these agents:
- the Support Lead
- the Internal Knowledge Researcher (Hatch)
- the Communications Writer (You've Got Mail)
- the Quality Reviewer (Kansa)
Flagged for review if:
- either of those facts changes
- any of those agents is changed
- nobody checks it again within 90 days
A flagged golden example is left out, so your agent team doesn't learn from it and Evy doesn't grade against it. A flagged golden case is left out of reruns, so it can't block a fix. Both stay out until you check them.
What you do when one is flagged
- Ask Evy, in its own thread, what to do: keep it, change it, or remove it.
- Decide, and tell your Chief in your own words. Your Chief saves it.
We show you: the Hattie B's practice inbox
If Hattie B's dropped Shut the Cluck Up, its hottest heat level, every golden example that mentions it would be flagged. The student would ask Evy about each one, decide, and tell the Chief.
Do this now: check one
In the same Chief thread, say:
For my first golden example (or, if I saved none, my first golden case), show me the facts and agents it depends on, and what would flag it for review.
What you should see
A note like the catering one above: the facts it depends on and where each is kept, the agents it depends on, and what would flag it. Usually nothing is flagged yet: it's new.
If you get stuck
| What you notice | What to do |
|---|---|
| Your golden cases have no notes | They still rerun. Ask your Chief to add a note to each one. |
| Nothing your team made this week is great yet | Save none for now. Add one when it comes. One honest example beats three you'd edit. |
| A note has no "where it came from" | Don't save it. Ask your Chief where it came from; if nobody knows, it doesn't go in. |
| You want to save a mistake you aren't sure about | Save it. A test that keeps passing costs only a little rerun time, and you can remove it later. |
| A golden example or golden case says "needs review" | It's left out until it's checked. Ask Evy, in its own thread, whether to keep it, change it or remove it. Then tell your Chief what you decided. |
| A new one says "needs review" straight away | A fact it depends on isn't in your library yet. Ask your Chief to get that fact added to your library, then check it again. |
| You'd like the team to copy an example word for word | That isn't what examples are for. They show the team what good looks like; each piece of work is still written fresh. |
What's in each note, and how a flagged one is reviewed: more detail on this lesson.
What you'll keep
Golden examples for your team, each saying where it came from, and a golden set with a note on each golden case, including a regression test for every mistake you chose to keep out.
Before you finish
Self-assessed. No gate.
Words you now own
- Golden examples: best-in-class results your team uses as its reference before similar work, and Evy grades against.
- Golden set: your team's standard exam: the same test jobs, each with a known right answer, rerun after every change.
- Golden case: one test in your golden set, with a known right answer.
- Regression test: a job added to your golden set after a mistake, so the mistake can't creep back in.
- Where it came from: who or what made it, and when.
- Needs review: a golden example or golden case set aside because a fact or an agent it depends on changed, or nobody has looked at it in 90 days.
Next: Turn on Evy's daily check.