universal-video-transcode/SKILL.md

---
name: universal-video-transcode
description: Use when transcoding video for iPad+Android playback.
version: 1.0.0
author: Hermes Agent
license: MIT
platforms: [linux, windows]
metadata:
  hermes:
    tags: [ffmpeg, hevc, h264, mp4, ipad, ios, android, transcode, nvenc, cfr, frame-timing]
    related_skills: [tubesync-selfhosted, local-audio-to-text]
---

# Universal video transcode — anything in, one MP4 that plays on iPad AND Android

One tool: `scripts/transcode.py`. Hand it a **path**; a file converts, a folder converts
everything in it. It reads the source, picks the recipe itself, encodes, and verifies the
result before accepting it.

```sh
python scripts/transcode.py "D:/video/clip.mkv"
python scripts/transcode.py "D:/video" --recursive --dry-run   # plan only
python scripts/transcode.py "D:/video" --out D:/converted
```

## Scope — read this before applying the size policy

This policy is for **H.264 (and older MPEG-4/DivX) sources**, where the source is fat enough that
HEVC at matched quality lands 25-50% smaller (the course-library case: 1080p H.264 at
550-2400 kbps, mp3 audio, files 300 MB-1.3 GB).

**Do NOT run the size hunt on YouTube-sourced libraries whose codec is VP9/AV1.** YouTube already
serves those at a near-optimal bitrate for their quality, so re-encoding an already-lean VP9/AV1
file can only grow it or lose quality - "the source is already as small as it gets, and that is
accepted". **YouTube's H.264 (AVC1) ladder is a different story, and worth measuring before
dismissing:** the compatibility encodes are fat (measured median bpp **0.053** on one library's
1080p AVC1 files, 2.4-4.7 Mbps), and they shrank to **34-67%** of the source video bitrate at
matched quality (min SSIM ~0.99) - see section 11 of the policy reference. Classify by bpp and
treat them like any other H.264 source; only the VP9/AV1 half is "no shrink sought".

| | H.264 / AVC1 source (this policy) | VP9 / AV1 source (YouTube) |
|---|---|---|
| goal | same look, **smaller** file | same look, same size - no shrink sought |
| quality gate | SSIM pinned + size ceiling (min ~0.99 is the usable frontier) | **per-frame content check vs the software decode** |
| source for a repair | the file itself | **the pre-conversion snapshot backup**, never the already-converted file |
| frame cadence | `-fps_mode cfr` | `-fps_mode cfr` |
| decode | any correct decoder | **never `*_cuvid`** (wrong frames, see below) |

## What "playable everywhere" actually means

The intersection that both platforms decode **without a third-party player**:

| | iPad / iOS | Android |
|---|---|---|
| Container | MP4/MOV only — **no Matroska** | MP4 and MKV both fine |
| Video | H.264 (all), HEVC `hvc1` (A9+) | H.264 Baseline mandatory; HEVC 5.0+ |
| Audio | AAC — **never Opus** | AAC and Opus both fine |

**Apple is the constraint, not Android.** iOS has no Matroska demuxer and cannot play Opus
at all, so MP4 + AAC is what forces the conversion. Android would have accepted the `.mkv`.

Two facts that decide the video codec:

- **HEVC must be tagged `hvc1`.** ffmpeg defaults to `hev1`, which Apple refuses. `-tag:v hvc1`
  is not optional.
- **Android only guarantees HEVC *Main profile Level 3*** (~SD). 1080p output is Level 4.0
  and 4K is 5.0 — fine on modern phones, but a low-end/old device may only reach Level 3.
  When the audience includes cheap Android hardware, prefer H.264 (`--codec x264`), which is
  mandatory from Android 3.0 and universal on iOS.

Full matrix, per-platform citations, and the HDR/10-bit caveats: **`references/compatibility-matrix.md`**.

## The three recipes it picks automatically

Decided from the **source file**, never from its filename or extension:

| recipe | when | video | audio |
|---|---|---|---|
| `copy` | H.264/HEVC with a usable timeline | `-c copy` — **bit-identical** | copy if AAC |
| `fix-audio` | same, audio is not AAC | `-c copy` | AAC 256k |
| `encode` | VP9/AV1, or H.264/HEVC whose timeline is unusable | HEVC (or x264) | copy if AAC, else AAC 256k |

`copy` is used whenever a copy is possible, because a copy is lossless **by definition** —
that is the strongest possible answer to "keep the quality the same".

## Quality policy ("as close to the original as possible")

- **Never upscale.** The source resolution is preserved; there is no `scale` in the pipeline.
- **HEVC** targets `-cq 27`, measured on this hardware at **SSIM 0.9937** (visually transparent)
  while landing at **~source size**. `-cq 28-30` buys 14-32% smaller at SSIM still >=0.992.
- **H.264** (when chosen for reach) targets `-crf 20`.
- **Audio**: copied when already AAC (bit-identical); otherwise **AAC 256k**, where the
  residual error is the *source's own* lossy error — 320k buys nothing measurable.
- **Bitrate cap**: `-maxrate` = 1.5x the source bitrate, `-bufsize` = 2.5x that cap. Never set
  `bufsize` equal to `maxrate` (a 1-second VBV buffer starves the encoder and collapses quality
  on complex scenes); never cap at exactly the source bitrate (that pushes quality *below* the
  original — a second-generation loss).

## The two flags that must never be dropped

```sh
-fps_mode cfr      # without it the encoder silently drops 0.5-1.3% of frames,
                   # leaving doubled frame intervals that play as judder
-pix_fmt yuv420p   # 8-bit 4:2:0; 10-bit / 4:2:2 / 4:4:4 break device compatibility
```

`-fps_mode cfr` makes ffmpeg *duplicate* a frame it cannot place instead of dropping it.
Duplicated frames cost almost nothing; a gap is visible. The tool adds both automatically and
fails the item if the output comes back non-uniform.

## Verifying "no dropped frames" — and whether a browser is needed

**Short answer: the browser is the arbiter, but it is NOT the gate. Automate the causal
check; use the browser only to spot-check.**

The user-visible symptom (choppy playback) comes from a *deterministic* property of the
container: whether the frame timestamps are in **presentation** order or **decode** order.
That is readable from the header, so it needs no playback at all.

The defect this matters for: a B-frame stream whose MP4 composition-offset table is
unusable — either a single constant `ctts` entry (`PTS == DTS + const`) or **no `ctts` box
at all** (`PTS == DTS`). The decoder still reorders by POC, so the timestamps attached to the
output frames are wrong and the compositor drops **~23%** of them — while
`r_frame_rate == avg_frame_rate` stays **EXACTLY** equal, so every fps-based test calls the
file healthy.

| gate | cost | covers HEVC? | role |
|---|---|---|---|
| `r_frame_rate` vs `avg_frame_rate` | ~50 ms | yes | catches dropped/irregular cadence only |
| **`ctts` / PTS-inversion structural check** | ~50 ms | **yes** | **the automatable gate — causal** |
| full decode (`ffmpeg -f null -`) | minutes | yes | decode integrity |
| real Chrome `droppedVideoFrames` | ~30 s/file | **NO** | arbiter for H.264 spot-checks |

Three traps that make the browser check unsuitable as an automated gate:

1. **Chrome has no HEVC decoder.** `canPlayType('video/mp4; codecs="hvc1…"')` returns `''`, so an
   HEVC file reports **0 decoded / 0 dropped** and *looks perfect while nothing played*. A
   0-frame result is **not** a pass. HEVC can only be confirmed on the Apple device itself.
2. It needs a real Chrome and a real page; ~30 s per file is far too slow for a library.
3. The measurement is statistical (a 1-frame drop is noise), so it needs a threshold, not equality.

The tool therefore runs the structural gate on **every** file automatically, plus a full decode
pass, and refuses the output otherwise. Measurements, the exact thresholds, and the
2x2 experiment that proved causality: **`references/verification-and-automation.md`**.

## Running it unattended (cron)

Safe to schedule, because it is idempotent and self-locking:

- **A lock file** (default `~/.transcode.lock`) refuses to start a second instance. This is not
  optional hygiene: two concurrent NVENC jobs share one encoder unit, starve each other, and
  **drop frames** — the exact defect the tool exists to prevent.
- **Idempotent**: a file that is already a playable MP4 is skipped, so a re-run costs a probe
  and nothing else. `--dry-run` prints the plan without encoding.
- **A failed item leaves the input untouched**, so the next run simply retries it.
- **Disk guard**: aborts below 5 GB free rather than filling the work drive (an ENOSPC
  mid-encode is reported as a misleading codec error and costs every remaining item).

```sh
# hourly; exits non-zero if anything failed
python /path/to/scripts/transcode.py "Z:/docker/tubesync/download" --recursive \
  --out "Z:/docker/tubesync/converted" >> /var/log/transcode.log 2>&1
```

For a TubeSync library specifically, prefer its own DB-driven pipeline (`tubesync-selfhosted`)
— it also updates the database rows. Use this tool for anything outside that library.

## Whole-library runs on a network share (in-place, safe)

`transcode.py` writes `dst + ".part.mp4"` **next to the source**. Over SMB/WebDAV/Dokan that
is the failure mode to avoid, and when the source is already `.mp4` then `dst == src`, so the
only copy of the file is the one being written. For a library run, drive ffmpeg directly
from a small runner instead:

1. copy the source **off the share** to local disk (chunked; treat a short read as failure)
2. encode locally, with the same mandatory flags (`-fps_mode cfr`, `-pix_fmt yuv420p`)
3. run the structural gate **and** a full decode pass on the local output
4. upload to `<name>.new.mp4` (never the final name)
5. read the share copy back end-to-end and compare **SHA-256 + byte count** with the local file
6. only then `os.replace()` it over the original, and re-probe the result on the share

Step 5 costs one extra network read (~1 h per 330 GiB) and is the only thing that actually
proves the remote copy is complete — `getsize()` cannot, and neither can a duration probe:
with `+faststart` the `moov` is written first, so a truncated file still reports full duration.
Keep a JSONL ledger and a per-file skip test ("already HEVC hvc1") so a reboot resumes safely.
A single instance only; the lock matters even more here because two runs would race on `dst`.

### `plan_for()` gaps to know before a library run

- **`mpeg4` (DivX/Xvid) returns `None` = "unsupported video codec"** — it is skipped, not
  converted, so `.avi`/`mp4v` files silently keep a codec no iPad can play. Give them an
  explicit re-encode.
- **Repairing a defective H.264 timeline returns `x264`, not HEVC** (hardcoded in the H.264
  branch). If the requirement is "everything in HEVC", force it — the recipe is a policy, not
  the tool's default.
- A truncated file (`moov atom not found`) is a **re-download**, not a transcode job; report it
  instead of retrying.

### Measured throughput (RTX 3060, hevc_nvenc p6, `-tune hq`, cq 27)

Use these to quote an ETA before starting — the estimate is the user's decision input.

| source | speed |
|---|---|
| 1080p25 H.264 | **6.4x realtime** (~160 fps) |
| 480x270 | ~34x realtime |
| per-file overhead | ~45 s (local copy + decode pass + upload + read-back hash) |

A 520 h / 324 GiB library is therefore ~73 h of wall clock, not the ~40 h a bitrate-based
guess suggests. Encode order is a real choice: newest-first finishes the currently-watched
content in hours instead of days.

## "Same quality" is not the same cq number (measured)

`hevc_nvenc -cq 27` and `libx265 -crf 27` are **not the same quality**. Same number,
different scale: on one 1080p25 source (source video 1.20 Mbps), 8-minute window, video only:

| setting | Mbps | % of source video | SSIM |
|---|---|---|---|
| source H.264 (x264 2-pass) | 1.20 | 100% | ref |
| hevc_nvenc cq27 | 1.31 | **109%** | 0.9971 |
| hevc_nvenc cq30 | 0.91 | 76% | 0.9963 |
| libx265 crf24 | 0.83 | 69.5% | 0.9963 |
| hevc_nvenc cq32 | 0.69 | 58% | 0.9955 |
| libx265 crf27 | 0.58 | 48% | 0.9952 |
| hevc_nvenc cq34 | 0.54 | 45% | 0.9947 |

**Compare only at matched SSIM, never at matched cq/crf numbers.** Matched pairs here:
NVENC cq30 vs x265 crf24 (both 0.9963) -> x265 9% smaller; NVENC cq32 vs x265 crf27
(0.9955/0.9952) -> x265 16% smaller. So software is **9-16% more bit-efficient at equal
quality**, not the 2-2.5x that a raw cq27-vs-crf27 comparison implies - most of that gap is a
quality difference. Price: ~3x realtime (x265 fast, 1080p) vs 6.4x (NVENC).

**Why a library can grow instead of shrink.** On sources that are already efficiently
encoded H.264, NVENC cq27 is a *higher* quality target than the source itself, so the video
comes back at ~109% and the file grows. Quality must be pinned to a number, e.g. "SSIM >= 0.996
against the source", which is the only self-consistent definition when the source is already
lossy and no master exists to measure against. SSIM >= 0.996 lands the video near 70% of the
source bitrate - that is where the actual saving lives.

**Classify the source before choosing a setting.** bpp = bitrate / (w * h * fps) measures how
much fat is left, and it is readable in milliseconds:

| bpp | meaning |
|---|---|
| < 0.015 | already soft - re-encoding gains almost nothing, prefer `-c copy` |
| 0.015-0.025 | lean (typical streaming) |
| 0.025-0.040 | normal |
| > 0.040 | real headroom - worth a real re-encode |

One library measured 131 files < 0.025 against 243 files > 0.040 out of 572 - a single global
cq cannot be right for both. Calibrate per class (1-2 representatives, 60-120 s sample, sweep 2-3
cq values, keep the cheapest that hits the SSIM target); per-file calibration costs 1-2 min each.
A `-maxrate` of 1.0x the source bitrate is a cheap hard guarantee that output never grows.

**Audio must match the source, not a fixed 256k.** A weekly series shipping mp3 at 128 kbps was
re-encoded to `-b:a 256k`: that added ~128 kbps for no audible gain, ~+10% file size on a
1.3 Mbps file - enough to cancel the video saving on its own. AAC is more efficient than mp3, so
`-b:a` = the source's own audio bitrate (clamped, e.g. 96-256k) is already a quality upgrade.

- **Never build a verification harness on a stream-copy cut.** `-ss X -c copy` on an H.264 file with B-frames loses packets (measured: a 60 s cut of a 25 fps source yielded 1270 frames instead of 1500, with the content shifted after each dropped packet), so an encode of that cut can never line up with anything and the comparison reports a fake SSIM 0.25 minimum. Compare whole files (or byte-copies of them): a byte copy is faithful, a stream copy is not. Corollary - before believing any picture verdict, make the harness prove itself on a file against ITSELF (must be lo=1.0).
- **Time-box a single-file mystery.** When one file resists diagnosis after a couple of experiments, park it (leave it untouched, log it as 'gate could not verify'), report it, and move on - a library run must not stall on one item. Revisit parked files in a batch later.

- **Killing a background session may not kill the process.** The session is a wrapper around the command; killing the session left the real python alive holding its lock, so the next start was refused by a lock that was telling the TRUTH while the log looked like a stale-lock bug. Check the real PID before concluding (`powershell -NoProfile -Command "Get-CimInstance Win32_Process -Filter 'ProcessId=<N>' | Select ProcessId,ParentProcessId,CreationDate,CommandLine"`) and kill that PID. Corollary, now implemented in `Lock`: the lock file is touched by a heartbeat every minute and staleness is decided by age (5 min) as well as pid, and a takeover that cannot remove a file still held by the OS overwrites it in place instead of crashing.

- **A watchdog that reads a PIPE can kill healthy work; send child stderr to a FILE.** ffmpeg with `-stats` writes a line every 0.5 s; a 64 KB pipe fills within minutes, ffmpeg blocks in `write()` on stderr, its OUTPUT stops growing - and a stall watchdog then kills a perfectly good encode. Measured: ~270 kills in one night, 24 files converted, encoder sitting at 0% with the process alive. Pipes are for bounded output only; a multi-minute encode goes to a log file, read back afterwards for the error text.
- **Watchdogs earn their keep: a driver update wedges GPU work silently.** The user updated the NVIDIA driver while a `hevc_nvenc` encode ran: the process stayed alive, output stopped growing, the encoder read 0%, and NOTHING reported an error - the tier sat frozen for 1.5 hours. Tell the user to pause GPU tiers before touching the driver, and keep an output-growth watchdog (14 min window) so it self-heals either way.
- **Metadata on a Dokan share is a CACHE, so verify installed files by content.** A rename (`os.replace`) did not land for one file while every ffprobe probe insisted it had: the run logged `ok 1085 MB -> 478 MB` and the live file was still the original H.264, with no error anywhere. After installing, read the file back and compare its SHA-256 with the encode; that check cannot be served from a cache. (Same share: `getsize()` lies, so never resume an upload from a share-reported size.)
- **Give concurrent runs their own temp files.** Tiers run side by side by design (copy = I/O, x265 = CPU, nvenc = GPU ASIC) and share the stage directory; fixed temp names let two gates overwrite each other's decodes and report a fake `100% of frames below 0.85 - systematic defect`. Tag every temp with the pid AND the file.
- **Separate the single-file path from the tree walk.** `--path FILE --dest TARGET --keep-source` runs one file anywhere on disk through the same tier rule, gates and hash-verified install. Watch out: `os.path.relpath` raises ValueError across drives (file on D:, library on Z:) - fall back to the absolute path, and the dest overrides where the install lands (all post-install checks must then probe the DEST, not the source).

## Pitfalls

- **Five-step pipeline, no exceptions**: decode → encode → `verify()` → `verify_content()` / SSIM gate → hash-verified install. A file is replaced ONLY when every step passed; a failure leaves the live file untouched.
- **Never trust this share's metadata**: read files back and compare SHA-256 (a rename can silently not land); never resume a write from a share-reported size.
- **Never build a verification harness on a stream copy or a seek**: compare whole files, decode both sides sequentially, and prove the harness on a file against itself (must be lo=1.0).
- **Judge a defect's SHAPE, not one frame**: mean < 0.98 or > 1% of frames below 0.85 fails; a handful of scattered low frames at a healthy mean is a fade, and SSIM on near-flat frames means nothing.
- **Kill by PID only**, one lock per tier, one NVENC job at a time, and pause GPU tiers before a driver update (an update wedges GPU work silently — the output-growth watchdog is the net).
- **A multi-day run must not be tied to the agent session.** Two tiers launched as Hermes background jobs with `persist_on_release` were both killed mid-encode the moment the user closed their session (measured 2026-10-08 10:37: `*_out.mp4` and `.err` frozen in the stage dir, no `PASS-DONE` in either log, ~40 min of work lost). Launch them **orphaned** instead — `powershell -NoProfile -Command "Start-Process -FilePath <bash.exe> -ArgumentList <script.sh> -WindowStyle Hidden"`, the script itself looping `library_run.py` with a retry — so the parent exits immediately and the agent's process cleanup cannot reach the tree. Verify the chain: the job's parent must be a bash whose own parent is already gone. Cost: no exit notification (watch the log) and **stopping is by PID only**.
- **The `copy` tier is not idempotent — never re-run a whole `--tier copy` pass.** Its only skip test is "already HEVC hvc1", and a copy output is H.264 by definition, so a second pass redoes every file it ever produced (193 of them here, ~40 GB of share I/O). For a handful of newly-arrived files use the single-file path per file: `library_run.py --tier copy --path <file> --keep-source`.
- **A re-do of an already-playable file earns a 0.975 mean bar; a first-time conversion does not.** Measured over one pass of this library: 11 attempts were rejected purely on the window mean (0.9655–0.9795) while accepted encodes had *worse* frame minima (0.33–0.90) — the bar was splitting a continuum, not separating defects (`0718 马斯克原理` failed with min 0.887 and **0.00 %** of frames below 0.85). A re-do exists only to reclaim an old setting's bloat from a file that already plays (HEVC hvc1 + AAC), and reaching a 0.98 mean costs more bits than the source's own bitrate — which the size ceiling forbids — so a strict bar strands the file with its bloat for ever while burning an encode every pass. `SSIM_MEAN_MIN_REDO = 0.975` therefore applies **only** when the live file is genuinely playable (HEVC hvc1 **and** AAC) and this run is replacing it from the snapshot original. The frame-share test (>1 % of frames below 0.85) is unchanged in both cases and remains the test that catches a real defect — under the relaxed bar `失控` (0.9655) is still rejected, and `0808 陌生人效应` (0.9739) keeps the full 0.98 bar because its live file is H.264 — not a re-do at all.
- **Full measured detail, every failure mode and its detector: `references/pitfalls.md`.**

## The owner's standing requirements — read this first

When the ask is "convert this library to HEVC, same look, same sound, playable on iPad+Android,
never bigger, no judder", the defaults here are **not** enough: a fixed `-cq 27` targets a quality
*above* an already-efficient H.264 source, so the library grows instead of shrinking.

| requirement | the answer |
|---|---|
| size | shrink where the source has headroom; growth is **acceptable** for already-compressed sources (YouTube-style) — playability is the goal there |
| same picture quality | pin it to a number: **SSIM >= 0.996** vs the source (video lands near 70% of the source bitrate) |
| same sound quality | copy AAC; for mp3/opus, AAC at **the source's own bitrate**, never a fixed 256k |
| iPad + Android | MP4 + `hvc1` + `yuv420p` + AAC |
| no judder | `-fps_mode cfr` on every re-encode, the three cadence defects gated, and **no `*_cuvid` decoders** |
| verification | mandatory, and it must assert on the output, not on the input |
| CPU encoding | allowed, but **tell the owner first** and quote the wall clock (~3x realtime vs 6.4x) |

Full policy, including the measured numbers, the source-classification table, the judder detector
table and every CPU/threading knob: **`references/hevc-conversion-policy.md`**.

## Support files

- `scripts/library_run.py` — the whole-library driver that enforces the policy above: bpp tiering
  (copy / nvenc / x265), `-maxrate` at 1.0x the source **video** bitrate + a size assertion, a
  per-frame SSIM gate with a one-step quality retry, audio matched to the source, local staging,
  SHA-256 read-back before an atomic replace, a JSONL ledger, a run lock, E-core pinning for the
  CPU tier, `--tier` to run tiers separately, `--dry-run` for the classification plan,
  `--keep-local` to pilot without touching anything, and `--origin-root` to redo files that an
  earlier bad setting bloated, sourcing them from a snapshot of the originals (never deleting it).
- `scripts/transcode.py` — the tool. `--dry-run`, `--codec`, `--cq/--crf`, `--audio`,
  `--channels`, `--cpu`, `--keep`, `--overwrite`, `--no-decode-check`, `--lock`.
- `references/compatibility-matrix.md` — per-platform codec/container facts with citations,
  the HEVC level caveat, and what breaks on each device.
- `references/verification-and-automation.md` — the defect classes, the structural gate, the
  browser arbiter and its blind spot, and the 2x2 causality proof.
- `references/hevc-conversion-policy.md` — the owner's standing requirements for a HEVC library job
  (size ceiling, SSIM-pinned quality, bpp classification, audio matched to the source, the three
  judder classes, the mandatory verification chain, and the CPU/throttling knobs, efficiency first).