Skip to content

Notes

Archive a YouTube channel, and keep it in sync

The four channel URLs that mean different things, the archive file that makes a nightly re-run cost nothing, and the rate limits that decide whether a 900-video archive finishes this week or not at all.

Archiving a channel is not a bigger version of downloading a video. It is a different problem, with three failure modes that only appear at scale: rate limiting, re-downloading what you already have, and a folder of a thousand files nobody can navigate.

All three are solved by flags. Here is the whole thing, then why each part is there.

yt-dlp "https://www.youtube.com/@CHANNEL/videos" \
       --download-archive archive.txt \
       -o "%(upload_date)s - %(title)s [%(id)s].%(ext)s" \
       -f "bestvideo[vcodec^=avc1][height<=1080]+bestaudio[ext=m4a]/best" \
       --write-info-json --write-thumbnail --embed-metadata \
       --sleep-requests 1.5 \
       --min-sleep-interval 5 --max-sleep-interval 15 \
       --ignore-errors --continue \
       --retries 5 --fragment-retries 20

First: which URL you point it at

A channel has several listings and they are not interchangeable. This is the most common way an archive comes out wrong.

URL What you get
/@CHANNEL The channel home — often only what the page shows, not everything
/@CHANNEL/videos Regular uploads. Usually what you want
/@CHANNEL/streams Past livestreams, which /videos excludes
/@CHANNEL/shorts Shorts only
/@CHANNEL/playlists Playlists, with duplicates across them

Livestreams are the trap. A channel that streams weekly can have more hours in /streams than in /videos, and pointing at /videos silently misses all of it. Archiving properly means two runs, sharing one archive file:

yt-dlp --download-archive archive.txt "…/@CHANNEL/videos" [flags]
yt-dlp --download-archive archive.txt "…/@CHANNEL/streams" [flags]

Count before you start:

yt-dlp --flat-playlist --print "%(id)s" "…/@CHANNEL/videos" | wc -l

--flat-playlist lists without opening each video. Seconds instead of minutes, and it tells you whether you are about to fetch 40 videos or 4,000.

--download-archive: the flag that makes this possible

Every completed video’s ID is appended to archive.txt. Every future run skips anything listed, without even fetching its metadata.

That one behaviour is what turns an archive into something maintainable:

  • Interrupted at video 340 of 900? Re-run. It resumes at 341.
  • Want new uploads only? Re-run the same command. It fetches what appeared since.
  • Rate limited today? Stop, re-run tomorrow. Nothing is lost.

Back up this file with the videos. Losing it means re-downloading everything, and on a large channel that is the difference between a five-minute sync and a two-day one.

Only successes are recorded. A video that failed on a temporary geo-block gets retried next time, which is what you want.

Naming, so it is still usable in a year

-o "%(upload_date)s - %(title)s [%(id)s].%(ext)s"
  • %(upload_date)s gives 20260731, so the folder sorts chronologically. Do not use playlist_index for a channel — YouTube’s ordering shifts, and the numbers stop meaning anything.
  • [%(id)s] keeps the video ID in the filename. It is how you find the original again after a title is edited, and how you cross-reference the archive file. Eleven characters, worth it.

For a channel with series, split by year:

-o "%(upload_date>%Y)s/%(upload_date)s - %(title)s [%(id)s].%(ext)s"

%(upload_date>%Y)s applies strftime to the date, so files land in 2024/, 2025/, 2026/. Anything past a few hundred files needs this.

Keeping the metadata

--write-info-json --write-thumbnail --embed-metadata

An archive of bare video files loses the description, the tags, the view count and the upload date the moment the channel goes. --write-info-json writes all of it next to the file as JSON. It is a few kilobytes and it is the difference between an archive and a pile of MP4s.

It is also queryable later:

jq -r 'select(.duration > 3600) | .title' *.info.json

Not getting blocked

This is what decides whether a 900-video archive finishes.

--sleep-requests 1.5
--min-sleep-interval 5 --max-sleep-interval 15

--sleep-requests pauses between metadata requests — enumerating 900 videos is 900 requests before much video arrives, and firing those flat out is the reliable way to earn HTTP Error 429 and lose the afternoon. The other two pause between downloads, randomised.

Adding roughly 15 seconds per video to a job that was already going to run for hours is not the cost it looks like. See 403, 429 and downloads that crawl for what happens without it — and for why a VPN makes it worse rather than better.

Deliberately absent: --concurrent-fragments. Speeding up individual downloads is exactly the wrong optimisation here. The goal is finishing, not being fast.

Format, and why not “best”

-f "bestvideo[vcodec^=avc1][height<=1080]+bestaudio[ext=m4a]/best"

For an archive of any size, H.264 at 1080p is usually the right trade: it plays in QuickTime and every editor without a special player, and it is a fraction of the size. In testing, one 4K video was 1.27 GB as 2160p VP9 and 245 MB as 1080p H.264 — across 900 videos that is the difference between a spare partition and a new drive.

If you are archiving because the source may vanish, take the best available instead and accept the size:

-f "bestvideo+bestaudio/best" --merge-output-format mkv

MKV because it accepts any codec combination without conversion. Why 4K files refuse to open in QuickTime is its own piece.

Running it on a schedule

Save it:

#!/usr/bin/env bash
# sync-channel.sh
set -euo pipefail
cd "$HOME/Archive/CHANNEL" || exit 1

yt-dlp "https://www.youtube.com/@CHANNEL/videos" \
       --download-archive archive.txt \
       -o "%(upload_date>%Y)s/%(upload_date)s - %(title)s [%(id)s].%(ext)s" \
       -f "bestvideo[vcodec^=avc1][height<=1080]+bestaudio[ext=m4a]/best" \
       --write-info-json --embed-metadata \
       --sleep-requests 1.5 --min-sleep-interval 5 --max-sleep-interval 15 \
       --ignore-errors --continue \
       >> sync.log 2>&1

chmod +x sync-channel.sh, then weekly rather than nightly — new uploads do not appear hourly, and each run costs requests whether or not anything is new.

One macOS-specific warning: a script run by cron or launchd gets a minimal PATH that usually excludes Homebrew, so yt-dlp and ffmpeg are both missing. Use absolute paths, or the job fails silently and you find out months later:

/opt/homebrew/bin/yt-dlp --ffmpeg-location /opt/homebrew/bin …

That failure mode is the silent-file problem in a different costume.

Before you archive 4,000 videos

Check the size. --flat-playlist gives the count; multiply by a realistic average. At 1080p H.264, 15 minutes is roughly 250 MB, so a thousand videos is around 250 GB. At 4K it is several terabytes.

And run it on twenty first:

yt-dlp --playlist-items 1-20 [all your flags] "URL"

Twenty videos tells you whether the naming works, whether the format selector resolves, and whether your sleep settings are enough. Discovering any of that at video 800 is expensive.

Honestly, on the app

This is one of the places where the command line is simply the right tool, and the app I build is not. Scheduled syncs, archive files, per-year folder templates — these belong in a script you can read and cron. The app is for pasting a link and getting a file.

The engine underneath is the same yt-dlp either way. Use whichever fits the job; there is no reason to pick one for everything.