YouTube Cairns Blueprint.
Watch 40 channels at once. Every new video gets transcribed, tagged, embedded, and dropped into your knowledge vault overnight. The 8D build that turns a list of YouTube channels into a source-grounded second brain, with the same three-tier shape every other Cairns build sits on.
- A working YouTube vault with L1 waypoints, dense cards in the L2 Chain of Density Card Catalog Layer (Card Catalog Layer for short) per source, and L3 raw transcript notes, generated from public captions on channels you actually care about.
- A searchable retrieval layer via Supabase pgvector, so your agent can ask questions across every transcript and cite the exact source instead of bluffing.
- An overnight ingestion pattern you can later wire to a cron job, so new videos flow in while you sleep and the vault stays current without you babysitting it.
Hand the blueprint to Claude.
Open Claude Code in your local workshop repo. Paste this, and Claude will read the blueprint end to end, ask you which channels you want to follow, and walk you through the build with confirmation gates between phases.
How this fits in the OS.
The Cairns Three Tiers framework. This build is one concrete instance of that contract, applied to public YouTube channels.
yt-dlp for captions, Python for the pipeline scripts, Obsidian as your local vault surface, Supabase pgvector for retrieval by meaning, and Infisical or 1Password for credentials.
A second brain that grows by itself overnight, source-cited answers across hundreds of hours of video, and a pipeline shape you can later point at podcasts, blogs, or any messy source pile.
What you'll build.
By the end of this build you have a local vault with three folders: cairns/L1/, card-catalog/L2/, and raw/transcripts/. You pick a public YouTube channel. The pipeline pulls a few captions, drafts a dense card per video, links the cards to ten durable waypoints, and embeds all three layers into Supabase so your agent can answer questions and cite the exact source.
The reference vault behind the public Build Lab lessons holds 56,621 transcripts across 48 channels at $0 external API cost, ingested over years using this same shape. You will not match that on a Sunday. That is fine. The Sunday goal is the shape working end to end on a small slice you actually care about. Scale is just more channels and more nights.
What changes for you.
The pipeline itself is straightforward Python. The bigger thing is the shift in how you treat YouTube as a knowledge source. Three internal shifts you'll feel:
- You stop bookmarking videos you'll never rewatch. The captions go into a vault you can query. "What did that channel say about Cairns retrieval?" becomes a real question with a sourced answer, not a search through your watch history.
- You stop dumping raw transcripts into chat windows. The agent reads waypoints first, picks a card, opens the raw transcript only to verify. Your context window stays small. Your answers stay grounded.
- You start thinking of overnight as compute time. Once the shape works, the next move is a cron job. Channels you follow get checked nightly. New videos land, get transcribed, get embedded, get summarized, get linked. By morning, the vault is current. You wake up to a brief, not a backlog.
Before you start.
This build assumes a few things are already in place. If any of these are unfamiliar, work through the prereq cards first.
- You understand the Cairns three-tier shape. If L1, L2 Chain of Density Card Catalog Layer, L3 doesn't ring a bell yet, read The Cairns Three Tiers first. The blueprint assumes you know the contract.
- Python 3.10 or newer. Standard library only. The pipeline scripts have zero dependencies beyond
yt-dlp. yt-dlpinstalled locally.brew install yt-dlpon macOS,pip install yt-dlpworks on Windows. The pipeline calls it as a subprocess, so all it needs is ayt-dlpcommand on your PATH. If Homebrew starts compilingrustordenofrom source (it does that on a macOS version it no longer ships pre-built packages for), stop it and runpipx install yt-dlpinstead. You do not need Xcode.- A Supabase project with the
vectorextension available. Free tier is fine for a Sunday build. You will run one SQL migration to create namespaced tables. - An OpenAI API key for embeddings, stored in your password manager. The blueprint defaults to
text-embedding-3-small. You can swap in a local model later, but the starter uses OpenAI so you can finish in one session. - Optional, only if you want graph traversal. Neo4j Desktop, Aura, or a local Neo4j server. The Sunday build does not require it.
- Public content only. Public YouTube channels, or channels you have rights to process. Do not point this at private uploads or content with usage restrictions.
Real API keys and service tokens never get pasted into Claude Code. Store them in Infisical or 1Password, with a placeholder like PASTE_HERE, and you type the real value directly into your password manager or your local .env file. If Claude offers to enter a real key for you, say no. You always do that part yourself.
The build.
Five phases. The starter prompt at the top of the page walks Claude through all of them with confirmation gates between each one. What follows is each phase in order, with what to paste into Claude Code and how to know it worked.
Configure your channel list and env
~15 minCopy the example files into the real ones. Pick one channel. Three videos max. You can scale up after you trust the shape.
In the local repo, copy blueprints/youtube-cairns/pipeline/channels.example.yaml to channels.yaml and pipeline/.env.example to pipeline/.env. Then ask me which public YouTube channel I want to start with. I'll give you the URL and a stable slug (the slug becomes part of the L2 Chain of Density Card Catalog Layer and L3 file paths, so pick something I won't want to rename). Edit channels.yaml with one channel and max_videos: 3. Do not write any real credentials into pipeline/.env. Leave the keys as YOUR_OPENAI_API_KEY_HERE style placeholders. I will type the real values in myself after we finish the dry-run pass.
pipeline/.env directly. They do not pass through this chat.
Pull captions and write the L3 raw layer
~10 min for 3 videosThe build.py script calls yt-dlp for each channel in your config, pulls public captions, and writes one immutable markdown note per video into vault/raw/transcripts/<channel-slug>/. These are your L3 source-of-truth files. Once written, you do not edit them. If a transcript is bad, you record the correction in L2 Chain of Density Card Catalog Layer later.
Run python3 blueprints/youtube-cairns/pipeline/build.py --config blueprints/youtube-cairns/pipeline/channels.yaml --vault blueprints/youtube-cairns/vault --max-videos 3 from the repo root. Show me what landed in vault/raw/transcripts/. I want to see three markdown files, each with a header block (video ID, channel slug, source URL) and a transcript body. If yt-dlp failed on any video, show me the stderr and we'll decide whether to skip it or troubleshoot.
The agent does this for you. You don't hand-download captions. The script handles the channel listing, the per-video caption fetch, the file naming, and the metadata header. You just confirm the L3 files look right.
Draft Chain-of-Density cards in the L2 Chain of Density Card Catalog Layer
~10 min for 3 videosPhase 2 gave you raw evidence. Phase 3 gives you the middle layer. The enrich.py script reads each L3 transcript and drafts one L2 Chain of Density Card Catalog Layer card per source. The card holds a dense summary, retrieval hooks, key entities, linked L1 Cairns, and the path back to L3.
Quick definition. Chain-of-Density is a writing pattern. The first pass is a sparse summary. The next pass keeps the same length but packs in more specific signal: entities, numbers, named ideas. You repeat until the card is dense without being longer. That dense card is what the agent reads at retrieval time.
Run python3 blueprints/youtube-cairns/pipeline/enrich.py --vault blueprints/youtube-cairns/vault. Then open vault/card-catalog/L2/sources/<channel-slug>/ in Obsidian and walk me through one card. I want to see: the dense summary, the retrieval hooks (the questions this video could answer), the linked L1 Cairns, the source path back to L3, and the promotion note (does this card suggest any L1 waypoint should change). The starter version is deterministic, so this is a draft I will review, not a finished card.
The starter cards are drafts. The production reference uses richer enrichment. For the Sunday build, the deterministic version is the right starting point because you can inspect what it did and adjust the prompt later. Treat every L2 Chain of Density Card Catalog Layer card as a draft until a human has read it.
Create your Supabase tables and embed the layers
~20 minThis phase moves your vault from local markdown to retrieval by meaning. Supabase pgvector is a column type in Postgres that lets you search by what something means, not just by the words it contains. You'll create namespaced tables for your vault, then embed every L1 waypoint, L2 Chain of Density Card Catalog Layer card, and L3 chunk into those tables. After this, your agent can ask a natural-language question and get back the matching cards and chunks with source citations.
What "embed" actually means. Every chunk of text gets converted into a list of numbers (an embedding) that represents its meaning. Two chunks about the same idea sit close together in that number space, even if they don't share any of the same words. The agent searches the number space, finds the closest matches, and reads the original text back.
Open blueprints/youtube-cairns/pipeline/supabase/migrations/0001_youtube_cairns_blueprint.sql. Replace __CAIRNS_PREFIX__ with the same value I'll set in pipeline/.env as CAIRNS_PREFIX (something like wade_youtube_cairns). Show me the edited SQL. Do not run it. I will copy it into the Supabase SQL editor in my browser and run it there myself. That keeps the service key out of this chat.
After the SQL runs in your browser, fill in the real values in pipeline/.env directly (Supabase URL, service key, OpenAI key, the matching CAIRNS_PREFIX). Then come back to Claude Code:
I've run the SQL in my Supabase SQL editor and filled in pipeline/.env with real values. Now run set -a; source blueprints/youtube-cairns/pipeline/.env; set +a followed by python3 blueprints/youtube-cairns/pipeline/ingest-rag.py --vault blueprints/youtube-cairns/vault --apply. Show me the row counts written to <prefix>_sources, <prefix>_chunks, and <prefix>_cairns. If any row count is zero, stop and we'll figure out why.
vector extension isn't enabled? In the Supabase dashboard, go to Database, Extensions, and enable vector. Then re-run the SQL migration. The match_* retrieval functions in the migration depend on the extension being live.
Query the vault and confirm citations
~10 minThe whole build exists for this moment. You ask a real question. The agent walks L1 to L2 Chain of Density Card Catalog Layer to L3 and answers with source citations. If the cascade works, the blueprint works.
Run python3 blueprints/youtube-cairns/pipeline/query.py "What are the strongest reusable ideas from this channel?" against my vault. Show me the answer plus the L1 waypoints it picked, the L2 Chain of Density Card Catalog Layer cards it pulled, and the L3 chunks it cited. I want to see source paths and video URLs in the citations. If the answer is generic or has no citations, that's a retrieval failure, not a good answer. Tell me which layer fell down.
Optional, only if you want graph traversal too. ingest-graph.py writes a Cypher file you can apply to Neo4j to get channels, videos, sources, entities, and Cairns as a navigable graph. Skip it on the Sunday build unless graph queries are why you started.
build.py + enrich.py + ingest-rag.py to a nightly cron and let the vault grow while you sleep.
When it works.
Success looks specific. If all of these are true, the blueprint is doing its job:
- L3 transcripts are immutable. You can re-run
build.pyand the same video does not get rewritten. The raw layer is a one-way ratchet. - L2 Chain of Density Card Catalog Layer cards each cite one L3 source. Every card has a path back to the transcript that generated it. No card invents content the transcript doesn't contain.
- L1 waypoints stay small. Ten Cairns. New videos can propose an L1 update through the card's promotion note, but L1 doesn't silently rewrite itself.
- Queries cite cards and sources. The answer to "what did this channel say about X" comes back with the L2 Chain of Density Card Catalog Layer card name and the L3 video URL. No citations means the retrieval cascade fell down somewhere, not that the answer is bad.
- Adding a channel is cheap. Edit
channels.yaml, re-run the three pipeline commands, the new content lands in the same three layers. No new code.
If you get stuck.
Common failure modes, ranked by how often they bite people:
- Tried to ingest 40 channels on the first pass. Most common. Start with one channel, three videos. Confirm the shape. Then scale.
- Edited an L3 transcript to "clean it up." Don't. The raw layer exists so the agent can cite evidence and you can audit the answer. Record corrections in L2 Chain of Density Card Catalog Layer or create a reviewed derivative.
- Skipped the L2 Chain of Density Card Catalog Layer review. Treated the deterministic enricher's output as final. The starter cards are drafts. Read them. Fix the obvious misses. The prompt improves from seeing the first few outputs.
- Pasted a real Supabase service key into the chat. Rotate the key immediately, then redo the env step with Infisical or 1Password and the placeholder pattern. If Claude offers to enter the real key for you, say no. Real values go into your password manager and into
pipeline/.envdirectly. - Query returns generic answers without citations. The cascade didn't reach L2 Chain of Density Card Catalog Layer or L3. Check that
ingest-rag.pywrote rows to all three tables (sources, chunks, cairns). If any count is zero, embedding failed for that layer. - Channel has weird caption coverage. Some channels have great auto-captions, some have nothing. If the channel you care about has poor captions, this blueprint is the wrong starting tool. You'd want a transcription pipeline (MacWhisper or Parakeet) feeding into
build.pyinstead.
Still stuck? Post the channel slug, the phase you're on, and the actual error, in the workshop Slack. Someone in the cohort hit the same thing.
Where to go next.
You shipped the first vault. The pattern compounds from here. The natural next moves on the shelf: