Your sources, in your library.
Name the channels, sites and accounts you learn from. Your agent pulls the latest from each into short notes you can read in a minute, with the full text kept on your computer. Free routes first; paid only when you say so and see the price.
Nothing for YouTube and websites. For X, Instagram or Firecrawl, do Keys, the First Time first.
You follow a few YouTube channels, a few websites, a few people on X. This blueprint gives your agent a list of them, called a pack, and how many of the latest items to pull from each. For every item it writes one short note: what it is, who made it, the link, a summary and a few quotes you can check. The full transcript or page goes beside it, on your computer only. A ledger remembers what it already pulled, so next week it pulls only what is new. YouTube and websites have free routes. X and Instagram cost a little, and nothing paid runs until you name a ceiling and say yes to the price.
A pack file that lists your sources, one note per item in library/sources/, the full text in library/originals/, and a ledger that stops duplicates.
What you learn from stops living in a feed. Your agent can search it, quote it with the timestamp, and research across it, because it is in your own files.
yt-dlp reads YouTube captions. A crawler, Firecrawl or the app's web fetch reads pages. Scrape Creators or Apify reads X. The ledger decides what is new.
You pick the sources and how many. You name a ceiling before anything paid, see each price first, and store any key in Infisical yourself. You read the notes.
Free first, then paid
Every platform has a free route except X. Paid is the add-on, never the default.
- YouTube. Free: yt-dlp reads the captions, with the free Exa connector as the backup. Paid: Scrape Creators, when YouTube keeps blocking the free route.
- Websites. Free: the r2c crawler (your agent checks whether your r2c skill can read a whole site), Firecrawl's free tier, or the app's own web fetch. Paid: a Firecrawl plan, when the free credits run out.
- X. No free route. Scrape Creators first, Apify second.
- Instagram. No free route. Scrape Creators or Apify, and transcripts can come back empty.
yt-dlp, the r2c crawler and your app's web fetch need no account. Firecrawl's free tier needs a free account and a key. Every paid route needs the provider's key, kept in Infisical.
YouTube
The free route: yt-dlp. A free command-line tool that reads a video's captions. It downloads the captions only, never the video.
- Install. Mac:
brew install yt-dlp. Windows:winget install --source winget --id yt-dlp.yt-dlp, orpy -3 -m pip install -U "yt-dlp[default]". Your agent says which and waits for your yes. - YouTube sometimes says slow down; your agent waits and tries again, up to three times.
- The backup: the Exa connector. Free; you sign in once in your app's connector settings, and it works best in a local session on your own computer. The free Exa connector can read YouTube transcripts; your agent checks on the first video that it got the spoken words.
When it's worth paying: Scrape Creators. Its transcript endpoint costs 1 credit a video, and is free when you ask for a cached copy. Worth it when yt-dlp keeps getting blocked, or for a big backfill.
Websites
The free route. Three ways, by what you want.
- A small public site, whole: the r2c crawler. Your agent checks whether your r2c skill can read a whole site. If not, it uses one of the other two. Plain Python, nothing to install. It reads a small public site from its sitemap; r2c's own notes give its limits and rules.
- A page that needs JavaScript, or a bigger site: Firecrawl's free tier. 1,000 credits a month, no card, about one credit a page. You make the free account and keep its key in Infisical.
- A page or two: the app's own web fetch. Nothing to set up.
When it's worth paying: a Firecrawl plan. Only when the free 1,000 credits run out each month. Your agent always sets a page limit before a crawl and shows you the credit count first. On the free tier there is nothing to approve; on a paid plan, it shows the price and waits for your yes, as with every paid call.
infisical run -- firecrawl scrape "PAGE_URL" -o library/originals/web/page.md # your key reaches the one command, never the chat
X
The free route. There is none that holds up. Your agent asks whether you want X at all before it sets anything up.
When it's worth paying.
- Scrape Creators first. Your agent may already know it from r2c. One key covers X, YouTube, Instagram and more. Pay as you go: the credits you buy never expire, there is no subscription, and a new account starts with 100 free credits and no card. A single post is 1 credit.
- Know this about accounts. Scrape Creators returns about 100 of an account's most popular posts, not its latest. For the latest posts from an account, use Apify.
- Apify second. The X actor checked on 10-17-2026 is Tweet Scraper V2, a community actor. On Apify's free plan you get 10 posts a run. On a paid plan it pulls at least 50 at a time, about 2 cents per 50. Your agent checks the actor is still current and its price before using it.
Credit the author in every note. Never train a model on what you pull. Respect X's terms. The full post goes in library/originals/x/, on your computer and off GitHub; the note keeps only a short quote.
Paid only, and short on purpose. Scrape Creators reads a reel's words for 1 credit if the reel is two minutes or less; longer reels cost more and get their own price check. Apify's Instagram Reel Scraper lists an account's reels and can add a transcript, charged by the minute of audio. Either way, transcripts can come back empty: music-only reels, no speech, or a reply that simply failed. Your agent keeps the caption, marks the note "no transcript" and moves on. It never fills the gap. The full caption and transcript go in library/originals/instagram/; the note keeps only short quotes.
Where everything lands
Three places in your workbench, plus the pack itself.
- One short note per item:
library/sources/<platform>/YYYY-MM-DD-<slug>.md. Title, author or channel, link, date published, date pulled, the route used, the cost if any, a summary, and three to five short quotes, with timestamps for video. Short quotes only. - The full text:
library/originals/<platform>/. The transcript, page, X post or Instagram caption, in full, as it came back. Your workbench keeps this folder off GitHub, so other people's full work is never pushed; your agent checks that in the first step. - The ledger:
library/sources/ledger.csv. One row per item: the link, platform, date pulled, route and note. This is what stops duplicates. - The pack:
work/source-packs/packs/<name>.md. The channels, sites and accounts, and how many of the latest from each. Running it again pulls only what is newer than the ledger.
Spending, only when you say
The default is $0. Three rules keep it that way until you decide otherwise.
- A ceiling for the task. Nothing paid runs unless you name one, like "up to $1 for this pack". It covers that one run.
- A price before every paid call. Your agent looks up the current price, works out the cost for exactly what you asked, shows it, and waits for your yes. Every time, whatever the amount.
- A hard cap at the provider. You also set a spending limit in each provider's own dashboard, and leave any automatic top-up off. That cap holds even if a run goes wrong.
Paid keys live in Infisical, and your agent runs each call through infisical run. Set it up once with Keys, the First Time; you run infisical init once at your workbench root, and it works for every folder inside.
infisical run -- scrapecreators balance # credits left, and the key never shows infisical run -- scrapecreators list # finds the exact command for an X post
What happens in the first pack
Nine checkpoints, each waiting for your yes. Here is what goes by so none of it is a stranger.
- It copies the blueprint to
work/source-packs/and checks your full-text folder stays off GitHub. - You name your sources and how many from each. It writes your pack file and shows it to you.
- It checks what you have: Python, yt-dlp, the r2c crawler, the Exa connector. It installs only what is missing, after your yes.
- One YouTube video, end to end: the note, the captions, the ledger row. Then one website the same way.
- X, only if you want it, with the price shown first.
- The whole pack, then the same pack again straight away, to prove nothing is pulled twice.
- The handoff: how to ask r2c a question across what you pulled.
yt-dlp --version # only if missing, it installs yt-dlp after your yes yt-dlp --ignore-config --no-playlist --skip-download --print "%(id)s|%(title)s|%(channel)s|%(upload_date)s|%(duration)s" "VIDEO_URL" # title, channel and date for the note yt-dlp --ignore-config --no-playlist --list-subs "VIDEO_URL" # picks the best English captions yt-dlp --ignore-config --no-playlist --skip-download --write-auto-subs --sub-langs en-orig --sub-format vtt -o "library/originals/youtube/DATE-SLUG.%(ext)s" "VIDEO_URL" # captions into library/originals/youtube/ python3 .claude/skills/r2c/scripts/crawl_public_site.py "SITEMAP_URL" --max-pages 10 --output library/originals/web/DATE-SITE.jsonl # only if your r2c has the crawler; the pages go to a file, not the chat # Codex: .agents/skills/r2c/, Windows: py -3 yt-dlp --ignore-config --flat-playlist --playlist-items 1:3 --print "%(id)s|%(title)s" "CHANNEL_URL" # the latest three, checked against the ledger
What you can ask your agent, from now on
Five asks that work once your pack exists.
Run my source pack. Pull only what is not in the ledger yet, free routes only, and tell me how many new notes you wrote for each source.
Pull this video into my library with the free route: [link]. Write the note with three key quotes and their timestamps, and add it to the ledger.
Add [channel, site or account] to my source pack with the latest 3, and tell me which route it will use and whether it costs anything.
Read my source notes from this month and total what I spent, by provider. Then check my credit balance with each provider I use.
Use r2c to answer [question] from the notes in library/sources/ and their originals. Answer with citations and timestamps. No saving.
Words you now own
- Pack. A named list of sources, with how many of the latest items to pull from each.
- Source note. The short file for one item: who, what, where, when, how it was pulled, a summary and quotes.
- Original. The full transcript or page text, as it came back. It stays on your computer.
- Ledger. The list of everything already pulled. The reason a second run pulls nothing twice.
- Caption track. One version of a video's captions, such as the English one.
- Route. The tool that pulled an item: yt-dlp, the crawler, Firecrawl, Scrape Creators, Apify. Every note records it.
- Ceiling. The most you will spend on one task. No ceiling, nothing paid.
- Credit. A provider's unit of use. Scrape Creators charges about one credit a request; Firecrawl about one a page.
Before you pull anything
Four habits the blueprint keeps, so your library is one you can trust and keep.
- Public content, for your own research. No logins, no paywalls, no private videos or accounts.
- Credit the maker. Every note names the author, channel or site, with the link.
- Never train a model on it, and respect each platform's terms.
- Read the notes. The summaries are drafts until you have read them. The quotes are copied word for word, so you can check them against the original.
You store each key in Infisical's dashboard yourself, and your agent only ever runs infisical run. If an agent ever asks you to paste a key, the answer is no.
Fair questions
Not unless you name a ceiling. YouTube and websites have free routes. X and Instagram are paid only, and you see the price before every call. For scale, the X actor above is $0.40 per 1,000 posts.
YouTube sometimes says slow down. Your agent waits and tries again, up to three times, then stops. The video gets no ledger row, so the next run tries it again. Try later, use the Exa connector, or pay one Scrape Creators credit for that video.
Not in this blueprint. Captions and text only: smaller, faster, and enough to search and quote with the timestamp.
No. YouTube Cairns builds a layered vault from YouTube channels. Source Packs collects from several platforms into short notes. r2c can then research them, or save them to Cairns if you use it.
Yes. The tools are the same. Pulling crosses the network, so Codex's sandbox asks before each pull. Say yes.