AI Build Lab / Learn · Agent Workforce · Week 4 · Module 03

Pick the problem and agree the fix

Of everything my agent team gets wrong, which one problem do I fix first, and what exactly will change?

Staff only · this page and its checks

Source: Internal's lesson, src/week4/internal/lessons/anw-w4-l05.md (Internal 60d90336, 2026-10-19 19:39), read by md_week4.py.
Copy: Internal's Markdown as published to the student shelf 10-12 (Internal 101e2662a = student 8275f21b; PR #215, receipt #218), vendored byte for byte by vendor_week4.py from the student-edition overrides; the published student tree is the same files without the "Staff only" blocks (checked on import). The drawings are the inline SVG figures in the lesson files, used as written.
Prompts: Word for word from the lesson file Internal ships (course/workforce/week4/). Not run end to end from Learn.
Checked in the app: Not checked in the app from Learn.
Open to students: Yes.
Other view: one part at a time (one part at a time is the lesson; one scroll is kept to compare)

Staff review only: full controlling prompt. Students paste only the short request below.

Use the native aibl-the-professor agent to guide the Week 4 lesson Pick the problem and agree the fix. Read course/workforce/week4/lessons/anw-w4-l05.md completely, its details page course/workforce/week4/reference/anw-w4-l05-details.md when asked for more, and the native aibl-course-guide and aibl-lesson-map methods for this app. You teach and read; you never group findings, pick the problem, write or change a fix, or approve anything for the student. The student talks to the Evaluation Analyst (Evy) and to the Operator (Gigawatt) in their own threads. Do not call, spawn or route to any agent. Do not edit any file. Never write, suggest or pass on an approval; every approval is the student's own words in Gigawatt's thread.

Use these words exactly. Say golden set, golden case, golden example and regression test; never saved tests or examples of great. A problem is one way the team's work goes wrong, seen in Evy's findings across the cases and aimed at one agent (agent level) or at the job as a whole (job level). One problem per round; all the fixes for that one problem may go in together, as one batch. A fix may touch more than one agent's instructions, the team's sources, and the Quality Reviewer (Kansa)'s watch list for the team. For a change to one agent's own instructions, Gigawatt uses the aibl-eval-improve-loop method and its files under runs/<job>/eval/<seat>/. The plan for a fix says what changes, why, which golden cases will be rerun, and what counts as success: the failing case now passes and nothing that passed before fails. The student approves the plan in Gigawatt's thread. Whether the change stays is decided later, in the lesson Make the fix, rerun the golden set, and keep or undo it. Say "approve the plan", never "two yeses". Agents stop only for the smoke alarm: sending outside, posting publicly, deleting, moving money, or changing account authority. Name agents job title first, nickname in brackets on first use; Evy is "it". Say "your work", never "business"; refer to other lessons by title, never by number.

The Hattie B's practice inbox is the worked example on the page. Never reveal or paraphrase any answer key for the practice emails; use only what the lesson page shows.

Ask open questions only. Never rank the problems for the student and never say whether a plan is good enough.

Part 1. If asked, read Evy's grouped findings the student shows, read only, and say back each problem, its cases, the rules it fails and the agent it points at or the job level.

Part 2. If asked, help the student weigh the problems with open questions: how often, how much harm, whether anyone would catch it first; the aim is the one fix that makes the biggest difference. The other problems are fixed later this week, the same way. The student picks. If Evy found no problems, explain the page's path: up to three harder cases (unused Hattie B's practice emails for the email team; cases Gigawatt builds with the aibl-synthetic-customer-builder method for a designed team), graded, saved and sealed like the others; if those pass too, the least certain rule becomes this round's problem.

Part 3. If asked, read Gigawatt's plan the student shows, read only, and check it names what changes, why, the golden set it will rerun, and what counts as success. Say plainly which of those is missing; never fill it in.

Part 4. If asked, explain what approving the plan allows and what it doesn't: nothing is installed until the student decides to keep the change.

Otherwise, if asked, explain the step and what the student should see.

Staff only: status, sources, what the build supports. Not student-facing.

Status: first draft, 8 October 2026, lane "lessons" (dispatch 51bbe9da), from spine v2 Module 3 and DESIGN-week4-2026-10-08.md section 6 (one problem per round, all fixes for it together; Evy writes findings grouped by problem, aimed at an agent or the job; Gigawatt proposes the batch; the owner's yes is door one). Two doors, plus the smoke alarm (Tyler, 8 October; constant in workforce/shed/design_approval.py).

Bound to the merged lanes.

Evy groups findings on the owner's direct ask (PR #202, "On demand").
The all-pass path uses aibl-synthetic-customer-builder (PR #205, owned by Gigawatt; it needs a team record and an approved design, so the email team adds unused practice emails from the pack instead).
Door one: aibl-record-team-change (PR #199) has the Charter Steward file the owner's yes as a ruling in work/course/staff/decisions/; the page says the yes is filed.
Agent-level fixes: Gigawatt uses aibl-eval-improve-loop with files under runs/<job>/eval/<seat>/ (STATE item 22). The multi-reviewer upgrade is the Week 3 thread's Friday PR; the page names neither the method nor its reviewers, so it stays true either way. The details page describes it.
The Hattie B's findings and plan below are a made-up worked example. "Pointing the researcher at the allergen line" follows the design's example; the pack holds that line in policies.md (Allergens).

Figure ID	Lesson	What it shows	Caption	Alt text
`w4-l05-two-doors`	`anw-w4-l05` Part 4	Five boxes left to right under two time labels (TODAY over the first; THE NEXT LESSON over the rest). The first and last are YOU boxes, highlighted: "Approve the plan" and "Keep it or undo it". The middle three are AGENTS boxes: "Gigawatt makes the change", "Your Chief of Staff reruns the golden set", "Evy grades the results", bracketed "The agents do this. You can walk away." Below, a tinted strip: "In between, agents stop and ask you only to:" with five chips (Send outside, Post in public, Delete, Move money, Change an account) and "For anything else, they make a sensible choice, note it for you to check, and keep going." Drawn by `l05_figs_svg.py`.	You approve the plan. The agents do the rest. Then you decide: keep it or undo it.	See the figure's title element.

In plain English

The Evaluation Analyst (Evy) has graded your cases, and you've compared grades. Today you read what Evy found, grouped by problem, and pick the biggest one. The Operator (Gigawatt) proposes a fix for it, and you say yes to the plan.

You fix one problem at a time. A fix can touch several things at once: an agent's instructions, the sources the team reads, and what the Quality Reviewer (Kansa) watches for. But it's all for one problem, so when you test it, you know what made the difference.

Nothing changes today. You approve the plan. In the next lesson, you ask Gigawatt to make the change, and you decide then whether it stays.

What you get

One problem chosen, and a fix for it you approved.

Why it matters

Change three things for three problems at once, and when something breaks you won't know which change did it.

How it works

The Course Companion (The Professor) guides you in its own thread. Evy groups Evy's findings by problem. Gigawatt proposes the fix and waits for you to approve it.

Your part

You pick the problem, read the plan, and approve it or ask for changes.

Before you start

You and Evy have compared grades. You did it in "Compare your grades with Evy's and teach Evy".
Nothing to open. Evy reads Evy's grades, and Gigawatt reads your team.
In Codex, start your first message to an agent in a thread with "Ask the <agent> to do this, and show me its reply word for word:". Start every message after that with "Tell the <agent> this, word for word, and show me its reply word for word:".

Who to ask. This lesson uses three agents. Each has one kind of job:

You want	Ask	Think of it as
Work done	Your Chief of Staff	Like a real chief of staff, it runs the day-to-day: it gets your team to do the work, like rerunning the tests.
Work graded	Evy	Your auditor. It checks the work and never does it, so its grades stay honest.
Your team changed	Gigawatt	Your builder. It changes how your team works, but only the way you approved.

Start or resume this lesson

Open your prepared workbench and call The Professor by its role name, aibl-the-professor. Paste only this.

Use the native aibl-the-professor agent to start the Week 4 lesson Pick the problem and agree the fix. Read course/workforce/week4/prompts/anw-w4-l05.txt and course/workforce/week4/lessons/anw-w4-l05.md. Ask me one question at a time and wait. I will talk to Evy and Gigawatt in their own threads.

The full instructions your agent follows for this lesson are in course/workforce/week4/prompts/anw-w4-l05.txt.

Part 1 of 4Read Evy's findings, grouped by problem· 10 min

Evy reads all its grades and gives you a short list of what's going wrong on your agent team. In the next part, you pick one thing to fix.

We show you: the Hattie B's practice inbox

Problem	Cases	Rules it fails	Level	Points at
Made-up food facts	The kids plate question	J1 · No made-up food facts; J4 · Nothing I'd have to change; A7 · Draft uses only the evidence; A8 · No guessing to fill a gap; A10 · Kansa catches unsupported facts	Job level and agent level	The Communications Writer (You've Got Mail) fills gaps; Kansa doesn't catch it
Promises only the owner makes	The cold-food complaint	J2 · No promises only I make	Job level	The writer
The wrong sign-off	Marcus	A9 · The practice voice's greeting and sign-off	Agent level only: the job still passed	The writer

Do this now: ask Evy for the problems

In a new thread with Evy, say this, with your team's name:

Group your findings for my <team name> team by problem. For each problem: the cases it shows up in, the rules it fails, and the agent it points at, or say it's at the job level.

What you should see

A short list of problems, like the one above. Each has the cases it shows up in, the rules it fails with their short names, and the agent it points at or "job level". If a fail doesn't fit any problem, ask Evy where it belongs, in the same thread.

If Evy found no problems

Three cases that all pass can't tell you much yet. Add up to three harder cases, then come back to this part:

Your email team: run practice emails you haven't used yet from the Hattie B's pack, such as the cold-food complaint or Marcus, through your Chief, the way you did last week. Grade them yourself and seal them, then let Evy grade them.
A team you designed: ask Gigawatt, in its thread, "Build two or three harder test cases for my <team name> team, labelled as made up, from my design." It builds them, your Chief runs them, and you grade them the same way.

If the new cases all pass too, your team is passing your rules on everything you've given it. Pick the rule you're least sure about, and make that the problem you fix: the fix is a reworded rule or a new golden case, and the next lesson proves nothing broke.

Part 2 of 4Pick the biggest one· 10 min

Pick the one problem that would make the biggest difference if you fixed it: to your work, to the people it's for, and to the risk you carry. You'll fix the others later this week, the same way.

Why only one? So you can see exactly what your fix changed. Fix three things at once, and if something breaks, you won't know which fix did it.

How to judge the difference

For each problem, ask:

How often does it happen?
How much harm would it do if it got out?
Would anyone catch it first?

The problem that happens most, does the most harm and slips through unnoticed is the one with the biggest impact. Fix that one.

We show you: the Hattie B's practice inbox

	Made-up food facts	Promises only the owner makes	The wrong sign-off
How often?	Once	Once	Once
How much harm?	A wrong diet or allergen fact can make someone sick	Commits the owner to a refund they never agreed to	Off-voice, but harmless
Would anyone catch it?	No: the customer would trust it	Only after the customer has the promise	Hardly anyone would notice
Decision	Fix now	Fixed later this week	Fixed later this week

All three happened once, so how often didn't decide it. Fixing made-up food facts makes the biggest difference: it could hurt someone, and nobody would catch it first.

Do this now: pick one

In the same Evy thread, say which problem you're fixing first, and why, in your own words, in as much detail as you need.

What you should see

Evy says back the problem you picked and your reason.

Part 3 of 4Read the fix Gigawatt proposes· 10 min

Now you take the problem to the Operator (Gigawatt). Gigawatt is the agent that builds and changes teams in your organization. Evy finds problems but never changes your team, the same way an auditor checks the books but doesn't keep them.

Gigawatt writes a plan for the fix, and stops there. Nothing on your team changes yet. You read the plan, and if it's what you want, you approve it in the next part. In the next lesson, you ask Gigawatt to make the change.

What a plan has in it

What changes. Each thing, and which agent or file it's in.
Why. Tied to the cases where the problem showed up.
What gets rerun. Every golden case, not just the one that failed.
What counts as success. The failing case now passes, and nothing that passed before fails.

We show you: the Hattie B's practice inbox

Three agents had a hand in the made-up fact. The researcher didn't find the answer, the writer filled the gap, and Kansa didn't catch it. So Gigawatt's plan has one change for each, all for the same problem. Here's the plan as Gigawatt shows it, in the four parts above:

Plan: fix made-up food facts on the email team

1. What changes
- The writer's instructions: add "If the evidence package marks a fact as missing, leave it as a gap and point the reader to the allergen line and a manager."
- The Internal Knowledge Researcher (Hatch)'s sources: also look in the allergen section of the policies, not just the menu guide.
- Kansa's watch list: watch for any diet or allergen fact that isn't in the menu guide.

2. Why
On the kids plate question, the researcher didn't find the answer. The writer filled the gap with "the fries are vegetarian", and Kansa marked the draft ready.

3. What gets rerun
All five practice emails. The fake bank email three times, because it must be refused every time.

4. What counts as success
The kids plate question passes. The other four grade the same as before or better.

Nothing changes until you approve it.

Do this now: ask Gigawatt for the plan

In a new thread with Gigawatt, say this, with your team's name and the problem you picked:

Evy found this problem on my <team name> team: <the problem, in your words>. Propose one fix for it: what changes and where, why, every golden case you'll rerun, and what counts as success. Don't change anything yet.

What you should see

A plan with all four parts above. If one is missing, ask Gigawatt for it in the same thread. Ask about anything you don't follow. If a change is for a different problem, ask Gigawatt to take it out and save it for later this week.

Part 4 of 4Approve the plan· 10 min

You approve the plan in your own words. Then the agents do the rest.

You approve the plan.The agents do the rest. Then you decide: keep it or undo it.

You approve the plan. The agents do the rest.

You approve the plan today. Then the agents make the change and test it. In the next lesson, you look at the results and decide: keep it or undo it.

Nothing changes without your OK, and nothing stays until you've seen it work.

Everything in between is the agents' job. They stop only if something would send outside your organization, post in public, delete, move money or change an account. For anything else, they make a sensible choice, note it for you to check, and keep going.

Do this now: approve it, in your own words

When the plan is what you want, approve it in the same Gigawatt thread, in your own words. For example:

I approve that plan for <your problem>. Go ahead.

What you should see

Gigawatt repeats the plan it's approved to make, with today's date. Your approval is saved, so there's a record of it. In the next lesson, you ask Gigawatt to make it. Nothing has changed on your team yet.

If you get stuck

What you notice	What to do
Evy found only one problem	Fix that one.
Evy found no problems	Follow "If Evy found no problems" in Part 1: up to three harder cases, then pick.
Two problems seem equally big	Pick either one. You'll fix the other later this week.
Gigawatt's plan fixes two problems	Ask Gigawatt to take out the second and save it for later this week. One problem at a time.
The plan changes an agent you didn't expect	Ask why, in the same thread. A fix can rightly touch the agent that should have caught the mistake.

How Evy groups problems, what a fix can touch, and how Gigawatt changes an agent's instructions: more detail on this lesson.

What you'll keep

One problem chosen with your reason, and Gigawatt's plan for fixing it, approved by you.

This is the one time you carry a problem by hand, so you see every step. After you turn on Evy's daily check ("Turn on Evy's daily check"), Evy hands each problem it finds to your Chief of Staff, and the fix comes to you ready in your brief, prepared by whoever's job it is. You still say yes first, and keep or undo after.

Before you finish

Before you finishFour checks

I read Evy's findings grouped by problemI picked one problem and said whyGigawatt's plan names what changes, why, what gets rerun, and what counts as successI approved the plan in my own words, and nothing changed yet

Self-assessed. No gate.

Words you now own

Problem: one way the work goes wrong, seen across your cases.
Fix: every change for one problem, made together.

Next: Make the fix, rerun the golden set, and keep or undo it.