Turn on Evy's daily check
How do I find out every day how my teams did, without grading everything myself?
- Source
- Internal's lesson, src/week4/internal/lessons/anw-w4-l08.md (Internal 60d90336, 2026-10-19 19:39), read by md_week4.py.
- Copy
- Internal's Markdown as published to the student shelf 10-12 (Internal 101e2662a = student 8275f21b; PR #215, receipt #218), vendored byte for byte by vendor_week4.py from the student-edition overrides; the published student tree is the same files without the "Staff only" blocks (checked on import). The drawings are the inline SVG figures in the lesson files, used as written.
- Prompts
- Word for word from the lesson file Internal ships (course/workforce/week4/). Not run end to end from Learn.
- Checked in the app
- Not checked in the app from Learn.
- Open to students
- Yes.
- Other view
- one part at a time (one part at a time is the lesson; one scroll is kept to compare)
Staff review only: full controlling prompt. Students paste only the short request below.
Use the native aibl-the-professor agent to guide the Week 4 lesson Turn on Evy's daily check. Read course/workforce/week4/lessons/anw-w4-l08.md completely, its details page course/workforce/week4/reference/anw-w4-l08-details.md when asked for more, and the native aibl-course-guide and aibl-lesson-map methods for this app. You teach and read; you never turn the daily check on or off, record a ruling, give a brief, undo a watch-list item or change Evy's notebook for the student. The student talks to the Chief of Staff in its own thread. Do not call, spawn or route to any agent. Do not edit any file. Never write, suggest or pass on a yes; every yes is the student's own words in the Chief's thread.
Use these words exactly. Evy's daily check is the Evaluation Analyst (Evy)'s daily pass: on the first brief of each day, while the student's ruling says it is on, Evy grades the work every recorded team did since the last check, on its own, and a dated folder named for that day and evy-daily under runs/ records that it ran. The ruling is the student's decision, filed by the Charter Steward in work/course/staff/decisions/, the same way the Charter Steward's daily check was turned on in Week 3. Before the brief, the Chief of Staff first runs the catch-up check: it checks the date and time, lists what should have run and what was missed, and sorts each missed item (run once now when only the latest run matters; replay each missed day in order when every run counts; skip and log a time-bound job that is now stale; never run late a job that sends or posts, which waits in Needs you for the owner), each run keyed to its slot so nothing runs twice. Evy reads the teams' saved work under runs/, a local copy of the Langfuse records, and a record of the owner's review chats with the Chief in both apps from a fixed script that keeps no one's words, only what happened (who acted, handoffs, sends, verdicts, and that a correction, a hold or a clean send happened), hides secrets, email addresses and phone numbers, and stays on this computer. A rule the owner wants kept reaches Evy's notebook only when the owner states it plainly ("That's a rule for the email team: ...") and the Charter Steward files it as a ruling; Evy adds it to its notebook from that ruling on the next check. The brief gains a short section, "How your teams did yesterday", one part per team: what ran, what passed, what failed and why, what was added to the Quality Reviewer (Kansa)'s watch list with its one-word undo, what the owner corrected that is now in Evy's notebook, and one suggestion. A bigger weekly review comes on Fridays, about fifteen minutes. Out of the box the check runs when the student asks for their first brief of the day; in the lesson on scheduling the student can give it a set time, and the morning digest then starts it at that time, with the first brief still the fallback. Name agents job title first, nickname in brackets on first use; Evy and Kansa are "it". Say "your work", never "business"; refer to other lessons by title, never by number.
The Hattie B's practice inbox is the worked example on the page. Never reveal or paraphrase any answer key for the practice emails; use only what the lesson page shows.
Evy grades every job each day, including the owner's sample. The counting script picks jobs for the owner to grade: every job at first, then one in two, then one in five, as they keep agreeing. Explain that the brief lists each under Needs you as "grade this one: <job>". Until the owner saves a grade with The Professor and asks the Chief to seal it, Evy's verdict shows "held until you grade it" wherever it could appear, including the dashboard, brief, daily files and Langfuse. The owner grades only rules they looked at. Evy never sees the owner's grade before its own is in. A job whose Evy answer the owner already saw cannot enter the sample. Once the Chief seals the owner's grade, the comparison runs at once, shows Evy's verdict beside the owner's reasons, counts their agreement, and notes on the seal that Evy graded first and its answer was hidden. Remind the owner: "Don't open Evy's folder for a job you're about to grade."
With Evy's daily check on, fix preparation is on too (the same ruling; "Turn off fix preparation" turns it off on its own). Evy hands each problem it finds up to the Chief as a problem card, a report, never an order; the Chief never tells Evy what to grade. The Chief picks the kind of problem from a fixed list and writes down why, and a script sends it to one seat: an agent's instructions, skill or tool to the Operator (Gigawatt); a wrong library fact to the Knowledge Librarian (Sallee); an outside fact to the External Researcher (Scratch), then Sallee; a rule read two ways to Gigawatt, with the owner approving the words and the Chief filing the new rule sheet; an unclear cause to the Evidence Pattern Analyst (Mitch Please) first; something too big to the Architect (Archie); money or the owner's policy to nobody, because it is the owner's call. The seat stages the fix in its own folder and nothing else changes. Every fix waits for the owner's yes; then the golden cases rerun and Evy grades them. Nothing is kept because it passed: a fix stays only on the owner's words to keep it, and the owner can undo it. "Yes F4, keep it if everything passes" is the owner's words for that one fix: the owner answers once, and if anything fails the Chief asks again (on unless the owner's ruling turns it off). A fix that touches a job held until the owner grades it waits and shows only "held until you grade it".
Ask open questions only. Never decide for the student whether to turn the check on, and never undo or keep a watch-list item for them.
Part 1. If asked, explain the order of the morning from the page: catch-up, the Charter Steward's daily check, Evy's daily check, then the brief. Explain the "grade this one" sample and when the owner may see Evy's answer.
Part 2. If asked, explain what Evy reads from the review chats, that it keeps no one's words, and how a rule the student states reaches Evy's notebook, from the page.
Part 3. If asked, read the ruling the student shows, read only, and say back plainly what it turned on, when it was filed, and how to turn it off.
Part 4. If asked, read the "How your teams did yesterday" section the student shows, read only, and say back each part. Explain how to undo a watch-list item; the student says it in the Chief's thread.
Part 5. If asked, read the fix the student shows in their brief, read only, and say back who prepared it, what it would change and what yes and skip do. Never say yes, keep or undo for the student.
Otherwise, if asked, explain the step and what the student should see.
Staff only: status, sources, what the build supports. Not student-facing.
Status: first draft, 8 October 2026, lane "lessons" (dispatch 51bbe9da), from spine v2 Module 5 and DESIGN-week4-2026-10-08.md sections 4.1 and 8 (daily pass on the first brief, the Steward's daily-check pattern, ruling turns it on, dated folder runs/<date>-evy-daily/; readers export_traces.py (ported from Camp's export-langfuse-traces) and transcript_digest.py; the catch-up check against the schedule ledger (schedule.yaml in work/course/staff/); the "How your teams did yesterday" section; the weekly review on Fridays; the daily pass stays local).
Bound to PR #202 where it has landed. Evy's daily pass: on the first brief while the owner's ruling has it on; runs/<date>-evy-daily/ (the daily pass script's daily.json, and Evy's teams-today.md) marks that it ran; on the owner's direct ask Evy grades what they name (the stuck table uses that for a brief that came before the ruling). The daily summary's watch line is kansa_watch.py's printed line (PR #199). The readers (export_traces.py, transcript_digest.py) and the catch-up ledger are the Readers and Daily lane's, not yet on a branch I could read; the page describes them in plain words.
- Turning it on: the page follows the Week 3 pattern (a ruling the Charter Steward files, from one plain yes in the Chief's thread). Integration (9 October): the ruling's decision line must read exactly "Evy's daily pass is on" (or "is off");
daily_pass.pymatches those words, and the Steward's duty 8 says so. - The order of the morning (catch-up, Steward check, Evy's check, brief) follows design section 8. Re-check against the Daily lane's
aibl-briefStep 0 change. - "Evy may add to its notebook and Kansa's watch list on its own, shown in the brief with a one-word undo": Tyler ruled this automatic (STATE items 6 and 7).
- The Hattie B's brief section below is a made-up worked example. Its watch-list line is the design's own wording.
| Figure ID | Lesson | What it shows | Caption | Alt text |
|---|---|---|---|---|
w4-l08-morning | anw-w4-l08 Part 1 | Four boxes top to bottom: you ask for your first brief; catch-up check (what was missed); the Charter Steward's daily check, then Evy's daily check; your brief with "How your teams did yesterday"; a side note "Fridays: the weekly review" | Your first brief of the day now starts with a catch-up and two checks. | When you ask your Chief of Staff for your first brief of the day, it first catches up on anything missed while your computer was off, then the Charter Steward checks your organization's records and Evy grades yesterday's work. Your brief then opens with how each team did. On Fridays there's a longer weekly review. |
So far you've asked the Evaluation Analyst (Evy) to grade. From today, Evy grades without being asked, every day. When you ask your Chief of Staff for your first brief of the day, Evy has already graded what every team did since the last check, and your brief tells you, in a few plain lines per team, how each one did.
For jobs picked for you to grade, your brief holds Evy's answer until your grade is saved and sealed.
When you want your team to remember something from now on, say it to your Chief plainly, as a rule: "That's a rule for the email team: don't offer a refund, ask me first." The Charter Steward files it as a ruling, and on the next check Evy adds it to its notebook for that team, without you filling anything in.
It runs at your first brief, or at a time you pick. Out of the box, the check runs when you ask for your first brief of the day, the same way the Charter Steward's daily check does. In lesson 12 you can give it a set time, like 7:30 AM before the Hattie B's prep call. Your morning digest then starts it at that time, and your first brief is still the fallback.
A short section in your first brief each day, one part per team, and a longer review each Friday.
You find out the day something slips, and every correction you make teaches your team for next time.
The Course Companion (The Professor) guides you in its own thread. Your Chief of Staff runs the catch-up and gives your brief. Evy grades without being asked. The Charter Steward files your ruling to turn it on.
You turn it on with one yes, then read your brief each day and undo anything you don't want.
Before you start
- Your shelves are saved. You did it in "Save your golden examples, and turn mistakes into regression tests".
- Nothing else to turn on first. If you finished "Get ready for Week 4", the Charter Steward's daily check is already on. If you only finished your email team, you turn it on in Part 3 of this lesson, together with Evy's.
- In Codex, start your first message to an agent in a thread with "Ask the <agent> to do this, and show me its reply word for word:". Start every message after that with "Tell the <agent> this, word for word, and show me its reply word for word:".
Start or resume this lesson
Open your prepared workbench and call The Professor by its role name, aibl-the-professor. Paste only this.
Use the native aibl-the-professor agent to start the Week 4 lesson Turn on Evy's daily check. Read course/workforce/week4/prompts/anw-w4-l08.txt and course/workforce/week4/lessons/anw-w4-l08.md. Ask me one question at a time and wait. I will talk to my Chief of Staff in its own thread.
The full instructions your agent follows for this lesson are in course/workforce/week4/prompts/anw-w4-l08.txt.
You see the order of things each morning. You get what to expect when you ask for your brief.
Your first brief of the day now starts with a catch-up and two checks.
Catching up when your computer was off
Your agents only work while your computer is on and the app is open. So before anything else, your Chief checks the date and time, and what should have run since your last brief. Anything missed is sorted: a job where only the latest run matters runs once now; a job where every run counts catches up day by day, in order; and a job that's too late to be useful is skipped and noted. Nothing runs twice.
Two checks, then your brief
The Charter Steward's daily check runs first: it checks your organization's records. Then Evy's daily check grades yesterday's work for every team, against each team's rules, the same way it graded your cases this week. Then your brief.
A team with no rule sheet is held, not graded; write its sheet to start Evy's grading.
Grading a job yourself, now and then
Evy grades every job each day, including the ones you grade yourself. The counting script picks a few for you: every job at first, then one in two, then one in five, as you and Evy keep agreeing. Your brief lists each one under Needs you as "grade this one: <job>". You don't see Evy's answer until you've saved yours and your Chief has sealed it. Wherever its verdict could appear, on your dashboard, in your brief, in the daily files or in Langfuse, you see "held until you grade it" instead. Evy never sees your grade before its own is in, and a job whose Evy answer you already saw cannot be picked for you to grade. Don't open Evy's folder for a job you're about to grade.
Ask The Professor, in its thread, to save your grade for only the rules you looked at. Then tell your Chief to seal it. The comparison runs as soon as the Chief seals it: Evy's verdict appears next to your reasons, counts toward how often you agree, and the seal notes that Evy graded first while its answer was hidden from you.
On the Hattie B's practice inbox, a new kids-plate follow-up email appears under Needs you as "grade this one: kids-plate follow-up email". The brief shows "held until you grade it" for Evy's answer. The student saves a grade with The Professor and asks the Chief to seal it. Only then does the student see Evy's grade beside their own reasons. This is a made-up example; no grade for that job is shown here.
The weekly review
On Fridays, your brief includes a longer review, about fifteen minutes to read: what ran, how much went out untouched or lightly edited or rewritten, which golden cases failed, how often you and Evy agree, examples waiting for review, and one change to make next.
Do this now: ask what should have run
In Claude Code: start a new thread, type @ and pick aibl-chief-of-staff. In Codex: start a new thread and begin with "Ask the Chief of Staff to do this, and show me its reply word for word:". Use this thread for the rest of the lesson. Say:
What is meant to run on a schedule in my organization, and when did each one last run?
You should see a short list, maybe just the Charter Steward's daily check. That list is what the catch-up works from.
You see how a rule you state reaches Evy, and what Evy reads from your chats with your Chief. You get corrections that teach your team without a form to fill in.
On the Hattie B's practice inbox
During the week, the student told the Chief: "That's a rule for the email team: always say which location you mean. There's more than one in Nashville." It wasn't on the rule sheet. The Charter Steward filed it as a ruling, with the student's words quoted. The next morning, Evy's daily check read the ruling, added it to Evy's notebook for the email team, and put it on the Quality Reviewer (Kansa)'s watch list. The brief said so, with a way to undo it.
What Evy reads, and what Evy never keeps
A fixed script reads your chats with your Chief, in whichever app you use (in both, if you use both). It keeps no one's words: not yours, not your Chief's, not a customer's. It keeps only what happened:
- who acted, and what was handed to which agent
- what was sent, and any review verdict
- that you made a correction, held something or sent it as it was, without what you said
It hides secrets, email addresses and phone numbers. What it keeps stays on your computer and never goes into a save point. That's why a rule you say in passing doesn't reach Evy. A rule reaches Evy's notebook when you state it as a rule and the Charter Steward files it.
Do this now: tell your Chief one correction
In the same Chief thread, tell your Chief one thing you'd want your team to remember from now on, as a rule, the way the student did. Say, in your own words:
That's a rule for the <team name> team: <your rule>. Have the Charter Steward record my ruling and show it to me word for word.
You'll look for it in your brief in Part 4.
You turn on Evy's daily check with one yes. You get a ruling, filed like your other rulings.
On the Hattie B's practice inbox
The student said "Turn on Evy's daily check for every team." The Charter Steward filed it as a ruling that reads "Evy's daily pass is on", dated 9 October, with the student's words quoted. A ruling is a decision of yours that your organization follows until you change it.
If you only finished your email team
You haven't met the Charter Steward's daily check yet. It runs just before Evy's. Each morning it checks your organization's records, like which agents are on which team, and puts any fix it suggests in your brief. It changes nothing without your yes. Turn both on together.
Do this now: turn it on
In the same Chief thread, say, in your own words:
Turn on Evy's daily check for every team, and the Charter Steward's daily check if it isn't on yet. Have the Charter Steward record my rulings and show them to me word for word.
What you should see
Your ruling, word for word, filed by the Charter Steward with today's date. If the Steward's daily check was off, you see a second ruling for it. To turn it off later, say "Turn off Evy's daily check" in your Chief's thread. The latest ruling wins, and your earlier rulings stay on file.
You ask for your brief and read how your teams did. You get your first daily section, and you decide what to keep from it.
On the Hattie B's practice inbox
How your teams did yesterday
Email team: 4 jobs ran. 2 passed. 1 failed: the cold-food complaint draft
offered a refund (the writer, rule J2). Evy's notebook: you said "always say
which location you mean", now a lesson for this team.
Kids-plate follow-up email: held until you grade it.
Added to Kansa's watch list for the email team: reply doesn't name the
location (W-002). Say 'undo W-002' to remove it.
Suggestion: fix the refund problem next, the same way you fixed food facts.
Needs you
grade this one: kids-plate follow-up email
Do this now: ask for your brief
In the same Chief thread, say:
Brief me.
Read the "How your teams did yesterday" section. If a team did no work since the last check, it says so in one line.
For each item added to Kansa's watch list, decide whether you want it. To remove one, say in the same thread:
Undo W-<number>.
What you should see
The section, one part per team, before the rest of your brief. If your computer was off for a day or more, the brief first says what was missed and what it caught up on.
In "Pick the problem and agree the fix" and "Make the fix, rerun the golden set, and keep or undo it", you carried one problem from Evy's grades to the Operator (Gigawatt) by hand. With Evy's daily check on, that carrying stops. The same ruling turns this on.
Evy keeps grading on its own. For each problem it finds, Evy writes a small problem card and hands it up to your Chief of Staff. That's a report. Your Chief never tells Evy what to grade. Your Chief decides what kind of problem it is and writes down why, and a script sends it to whoever's job the fix is:
| What the problem turns out to be | Who gets the fix ready |
|---|---|
| An agent's instructions, skill or tool | Gigawatt |
| A wrong or old fact in your library | The Knowledge Librarian (Sallee) |
| Something that changed out in the world, like opening hours | The External Researcher (Scratch) checks it, then Sallee |
| A rule you and Evy read two ways | Gigawatt rewords it; you approve the words, and your Chief files the new rule sheet |
| The cause isn't clear yet | The Evidence Pattern Analyst (Mitch Please) looks for the pattern first |
| Too big for one fix, like a new step or a new agent | The Architect (Archie) designs it |
| Money or one of your own policies | Nobody. It's your call |
Whoever gets it ready puts the fix in its own folder and changes nothing else. A script checks that. Your brief then says who made it and that nothing has changed yet. Each fix goes through the same two doors as before. Nothing changes until you say yes. Then the golden cases rerun, Evy grades them, and you see the before and after. Nothing is kept just because it passed: it stays only if you say to keep it, and you can undo it. If you say "yes F4, keep it if everything passes", you only have to answer once. If anything fails, it asks you again, and you can always undo it later. If a problem touches a job you're still grading, it waits until you've graded that job, and nothing about it shows before then.
On the Hattie B's practice inbox
Needs you
- Gigawatt has a fix ready for made-up food facts (F4). The kids plate reply
called the fries vegetarian; the menu guide doesn't say that. The writer will
leave a gap as a gap, the researcher also reads the allergen line, and Kansa
watches for any diet fact not in the menu guide. Nothing has changed yet.
- Sallee has a correction ready (F5), checked by Scratch: your hours card says
the Sunday kitchen closes at 9 PM; your website and your Google listing both
say 8 PM since Oct 1. Nothing has changed yet.
Outstanding
- Mitch Please is checking a pattern: the sign-off changed across 4 jobs this
week. No decision needed today.
Do this now: answer one fix
When your brief shows a fix ready, say in your Chief's thread:
Yes F4.
Or "Skip F4", or "About F4" to ask about it first, or "Send F5 to Gigawatt instead". With two ready, "Yes to both" applies them one after the other. When the result comes back, say "Keep" or "Undo".
What you should see
The fix made, the golden cases rerun, and Evy's before and after. Kept fixes are written into your team's change record, the same way as in "Make the fix, rerun the golden set, and keep or undo it". If Evy found nothing to fix, your brief has no fixes, and that's fine.
If you get stuck
| What you notice | What to do |
|---|---|
| Your brief has no "How your teams did yesterday" | It runs once a day, on the first brief after it's on. If you had a brief earlier today, before you turned it on, ask Evy in Evy's own thread: "Grade yesterday's work for every team now, and write today's daily summary." Then ask your Chief for your brief again. |
| It says a team did no work | That's fine. Evy grades only what ran. |
| Your app asks to run something during the check | Allow it. The check reads your files and saves its results in today's folder. |
| A correction you made isn't there | A correction said in passing isn't kept: the chat record keeps no one's words. Say it as a rule, in your Chief's thread: "That's a rule for the <team name> team: <your words>. Have the Charter Steward record my ruling." It reaches Evy's notebook on the next check. |
| Something was added to the watch list you don't want | Say "Undo W-<number>." The item is retired and its history is kept. |
| You want it off | Say, in your Chief's thread: "Turn off Evy's daily check. Have the Charter Steward record my ruling." |
| You want Evy's daily check on, but no fixes got ready | Say, in your Chief's thread: "Turn off fix preparation. Have the Charter Steward record my ruling." Evy keeps grading; you carry problems by hand again. |
| A fix says it's out of date | Something it touches changed after it was made ready. Nothing was applied. Say "Make F4 ready again." |
What the catch-up does, what Evy reads, and what the weekly review holds: more detail on this lesson.
What you'll keep
Evy's daily check, turned on by your ruling: a short section in your first brief each day, one part per team, a longer review each Friday, and fixes made ready by whoever's job they are, each waiting for your yes.
Before you finish
Self-assessed. No gate.
Words you now own
- Evy's daily check: Evy grading every team's work from the day before, run before your first brief.
- Catch-up: your Chief working out what was missed while your computer was off, and running what still matters.
- Weekly review: a longer look back each Friday at how your teams did.
- Ruling: a decision of yours, filed by the Charter Steward, that your organization follows until you change it.
- Problem card: Evy's short report of one problem it found, handed up to your Chief. Evidence, never an order.
- Fix ready: a fix someone has prepared in its own folder, waiting for your yes. Nothing has changed yet.
Next: Open your dashboard.