More detail: Make sure everything your agents do is recorded
Optional reading for this lesson. Everything you must do or decide is in the lesson.
This page goes with the Week 4 lesson Make sure everything your agents do is recorded (course/workforce/week4/lessons/anw-w4-l10.md). You don't need it to finish the lesson. Read it when you want the longer version.
What every record carries
| Field | What it says |
|---|---|
| Name | Which app made it: a Claude Code turn, a Codex turn, or the turn of an agent another Codex agent started |
| Session | The app's own conversation id |
| You | A short handle for you, never your email address |
| Environment | Course, or everyday work after you switch |
| Versions | The recorder's version, and a fingerprint of your agents' instructions |
| Job, team, agent | Read from the job your Chief opened |
| App and computer | Which app, its version, and which computer, by a short name, never its full name or a folder path |
| Parent | For an agent another agent started, the one that started it |
Tags are short and set when the record is made: the app, the environment, the team, the agent, "started by another agent" where it was, and the skill used.
How the job, team and agent get on the record
When your Chief opens a job, it writes a small note on your computer saying which job, team and agent are working. The recorder reads it and stamps each record.
The catch-up sweep
Some turns finish without the app telling the recorder. Every 15 minutes a sweep picks up anything missed. Each record's id is worked out from the app, the conversation and the turn, so a record the sweep sends again lands in the same place instead of twice.
The check, step by step
- Send one test record with a made-up secret planted in it.
- Read it back from Langfuse by its id.
- Confirm every field, the link to its parent, the model's usage, and that the made-up secret is hidden.
- Give you the link.
- Run the sweep and confirm nothing was sent twice.
A check is passed only at step 2. A record that was sent but never read back hasn't passed.
Living with a recorder you already have
Gigawatt looks for any Langfuse recorder already on your computer and tells you what it found. It never overwrites or removes it. It installs its own under its own names, adds its entries beside yours instead of replacing them, keeps its own state in its own folder, and, if you ever remove it, removes only its own entries.
The 30-day window
The free Langfuse plan keeps records for 30 days. Evy's daily check saves a copy of what Evy needs on your computer, so grading doesn't depend on that window.
Evy's grades in Langfuse
Evy finds the job's records in Langfuse. It uses the local copy on your computer for every quote in its grade. Once you ask the Chief of Staff to have the Charter Steward file "Evy writes grades to Langfuse: on", the Steward records the project and a fingerprint of the key. Gigawatt and Evy check that status without showing the key. A new project or new keys require the Steward to file the ruling again.
For each rule, Evy writes one score named evy.<team>.<rule>. PASS is True and FAIL is False. The score's comment names the rule, verdict, one-line reason, and a quote of no more than 25 words with its local file. An agent rule goes on that agent's last record for the job. A job rule goes on the job's first Chief of Staff record. HOLD is never written. There is no total or average score.
Evy reads every score back from Langfuse and sends it once more to check that none doubled. (Near midnight UTC it skips that second send, because a send on a new day could add a second row; the single score read back is the proof.) Its summary quotes the script's line, such as "Langfuse: 4 of 4 scores read back". A "sent" message is not proof. Evy does not delete or change a score, and it does not turn on paid automatic evaluators.
If a job is in your blind grading sample, Evy's scores wait until you save and seal your own grade. Then Evy's scores and your owner.<team>.<rule> scores go in together. Leave Evy's job folder closed until you have sealed your own grade. You can compare the two sets in Score Analytics. The saved agreement count remains the count of record.
The standing rule
When a new app, agent or computer appears, your Chief flags it in your brief. Gigawatt wires the recorder in, sets its tags, runs the same check, and the Charter Steward records it.