From 0cdb54f910b604ef3b98742b90a1137ebbcff72a Mon Sep 17 00:00:00 2001 From: Hermes Agent Date: Sun, 4 Oct 2026 13:01:14 +0800 Subject: [PATCH] universal-video-transcode: any file in, one iPad+Android-playable MP4 out Give it a path; it reads the source, picks the recipe, encodes, and verifies the result before accepting it. The problem it exists to remove: a file can drop ~23% of its frames in a browser while r_frame_rate == avg_frame_rate reads EXACTLY equal, so every fps-based check calls it healthy and it never enters a repair queue. The cause is deterministic and header-readable -- the MP4 composition-offset table is unusable, either a single constant ctts entry (PTS == DTS + const) or no ctts box at all (PTS == DTS) -- so the frame timestamps are in decode order and the decoder's reordered output frames carry the wrong times. - three recipes chosen from the source FILE, never the filename: lossless -c copy when the timeline allows it, AAC-only fix when just the audio is wrong, full HEVC (or x264 for reach) when the timeline cannot be copied - quality policy: never upscale; HEVC -cq 27 measured at SSIM 0.9937 ~= source size; audio copied when already AAC, else AAC 256k - the two flags that must never be dropped: -fps_mode cfr and -pix_fmt yuv420p - verified before acceptance: duration, resolution, codec, hvc1 tag, 8-bit, uniform cadence, no flat composition offsets, full decode pass, and a read-back after the write - built for cron: lock file (two concurrent NVENC jobs starve one encoder and DROP FRAMES), idempotent, leaves the input untouched on failure, aborts below 5 GB free - scripts/selftest.py builds both defect shapes and asserts the detector, including a Matroska case that must NOT be flagged Measured on the file that prompted this: 23.0% of frames dropped before, 0.0% after. --- .gitignore | 4 + AGENTS.md | 84 ++++ README.md | 119 ++++++ SKILL.md | 173 ++++++++ references/compatibility-matrix.md | 95 +++++ references/verification-and-automation.md | 124 ++++++ scripts/selftest.py | 174 ++++++++ scripts/transcode.py | 492 ++++++++++++++++++++++ 8 files changed, 1265 insertions(+) create mode 100644 .gitignore create mode 100644 AGENTS.md create mode 100644 README.md create mode 100644 SKILL.md create mode 100644 references/compatibility-matrix.md create mode 100644 references/verification-and-automation.md create mode 100644 scripts/selftest.py create mode 100644 scripts/transcode.py diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..4d1d86b --- /dev/null +++ b/.gitignore @@ -0,0 +1,4 @@ +__pycache__/ +*.pyc +*.part.mp4 +.transcode.lock diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..ffb91f0 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,84 @@ +# AGENTS.md — entry point for any AI session touching this repo + +**This repo is the canonical source of the `universal-video-transcode` Hermes skill.** +Read this file first, then `README.md`, then the specific reference you need. + +Owner: Lee Teong Hoe (Mr Hoelee) · me@hoelee.com · WhatsApp +60 12-797 2969 + +--- + +## 1. What this is + +One tool that turns **any** video/audio file into an MP4 that plays on **both** iPad and +Android, at the best quality the target codec allows. + +| layer | what | where | +|---|---|---| +| the tool | `scripts/transcode.py` — path in, verified MP4 out | this repo | +| the gate | `scripts/selftest.py` — builds both defect shapes and asserts the detector | this repo | +| the skill | `SKILL.md` + `references/` — loads into Hermes | copied to the skill dir (§4) | + +There are **no credentials** in this project, so there is no `SECRETS.md`. + +## 2. The non-negotiables (read before you change anything) + +1. **`-fps_mode cfr` on every video re-encode.** Without it the encoder silently drops + 0.5–1.3% of frames, leaving doubled frame intervals that play as judder. `verify()` + fails an item whose output comes back non-uniform — never remove that check. +2. **`-tag:v hvc1` on every HEVC output**, and `-pix_fmt yuv420p`. Apple refuses `hev1` + (ffmpeg's default) and non-8-bit formats lose device compatibility. +3. **Never copy a video whose timeline is defective.** `-c copy` cannot retime — ffmpeg will + not rewrite `stts`/`ctts` under stream copy — so a flat input survives into the output. + `plan_for()` decides this; do not "simplify" it into an unconditional copy. +4. **One encoder at a time.** The lock file is not optional hygiene: two concurrent NVENC + jobs share one encoder unit, starve each other, and drop frames. +5. **Run `python scripts/selftest.py` after touching the detector.** It is the only thing + standing between this tool and shipping files that stutter. +6. **Never write encode output straight to a network share.** Encode to local disk, verify, + then copy the complete file across; a streaming `.part` write leaves a corrupt stub on any + blip. If the destination is remote, keep the output local. + +## 3. Layout + +``` +SKILL.md the Hermes skill (installed per §4) +README.md human-facing overview +scripts/transcode.py the tool: plan -> encode -> verify -> keep input on failure +scripts/selftest.py proves the detector; both defect shapes, no false positives +references/compatibility-matrix.md per-platform codec/container facts with citations +references/verification-and-automation.md the defect classes, the structural gate, the 2x2 proof +``` + +## 4. Installing the skill + +```sh +cp -r . "$LOCALAPPDATA/hermes/skills/media/universal-video-transcode" +``` + +Keep this repo the source of truth and copy **to** the skill dir, not the reverse — edits +made only in the skill dir are lost the next time it is reinstalled. + +## 5. What "verified" means here + +An output is accepted only if all of these hold, each read back from the file: + +duration within 1% · resolution identical · expected video codec · `hvc1` tag on HEVC · +audio is AAC · 8-bit `yuv420p` · uniform cadence · **no flat composition offsets** · +full decode exits 0 · read-back after the write matches. + +Do not claim a fix without running the arbiter. `measure_playback.py` in the +`tubesync-selfhosted` skill plays a file in real Chrome and reports +`getVideoPlaybackQuality().droppedVideoFrames` — but note **Chrome cannot decode HEVC**, so +an HEVC file reports 0 decoded / 0 dropped and that is *not* a pass. Judge HEVC by the +structural gate and confirm on the Apple device. + +## 6. Related work + +- **`hoelee-tubesync-management`** — the TubeSync archive this tool was extracted from. It + owns the DB-driven pipeline (`tsconv.py` / `tsfix.py`) and the deeper frame-timing + forensics. `tsconv.flat_timing()` is that repo's copy of the detector: **keep the two in + step**, and note that the missing-`ctts` shape was once missed by three duplicated copies + of exactly this logic. One implementation per repo. +- `references/verification-and-automation.md` carries the measured 2x2 that proved the + defect is container timing and not the bitstream. If you are about to argue that a + bitstream property is the cause, read that first. diff --git a/README.md b/README.md new file mode 100644 index 0000000..9333fb5 --- /dev/null +++ b/README.md @@ -0,0 +1,119 @@ +# universal-video-transcode + +**Any video or audio file in, one MP4 out that plays on both an iPad and an Android +device — at the best quality the target codec allows.** + +Give it a path. A file converts; a folder converts everything in it. + +```sh +python scripts/transcode.py "D:/video/clip.mkv" +python scripts/transcode.py "D:/video" --recursive --dry-run # plan only, no encoding +python scripts/transcode.py "D:/video" --out D:/converted +``` + +## What it does + +It reads the **source file** (never the filename) and picks one of three recipes: + +| recipe | when | video | audio | +|---|---|---|---| +| `copy` | H.264/HEVC with a usable timeline | `-c copy` — **bit-identical** | copy if AAC | +| `fix-audio` | same, but the audio is not AAC | `-c copy` | AAC 256k | +| `encode` | VP9/AV1, or a timeline that cannot be copied | HEVC (or x264) | copy if AAC, else AAC 256k | + +A copy is used whenever a copy is possible, because a copy is lossless *by definition* — +the strongest possible answer to "keep the quality the same". + +Then it **verifies** the output and refuses to accept it otherwise: duration, resolution, +codec, `hvc1` tag, 8-bit pixel format, uniform frame cadence, no flat composition offsets, +and a full decode pass. + +## Why it exists + +A file can drop **~23% of its frames** in a browser while `r_frame_rate == avg_frame_rate` +reads **exactly** equal — so every fps-based check calls it healthy. The cause is +deterministic and header-readable: the MP4's composition-offset table is unusable, either a +single constant `ctts` entry (`PTS == DTS + const`) or **no `ctts` box at all** (`PTS == DTS`). +The timestamps are then in *decode* order, and the decoder's reordered output frames carry +the wrong times. + +This repo gates on that, structurally, without playing anything. + +```sh +python scripts/selftest.py # builds both defect shapes and asserts the detector +``` + +``` +fixture expect got verdict +src.mp4 False False ok healthy MP4, proper ctts +src.mkv False False ok Matroska - no ctts by design +flat_noctts.mp4 True True ok MP4 with NO ctts box +flat_ctts1.mp4 True True ok MP4 with a single constant ctts entry +``` + +## Compatibility + +| | iPad / iOS | Android | +|---|---|---| +| Container | MP4/MOV only — **no Matroska** | MP4 and MKV both fine | +| Video | H.264 (all), HEVC `hvc1` (A9+) | H.264 Baseline mandatory; HEVC 5.0+ | +| Audio | AAC — **never Opus** | AAC and Opus both fine | + +Apple is the constraint, not Android: iOS has no Matroska demuxer and cannot play Opus, so +MP4 + AAC is what forces the conversion. Two traps: HEVC must be tagged **`hvc1`** (ffmpeg +defaults to `hev1`, which Apple refuses), and Android only *guarantees* HEVC Main profile +**Level 3** — use `--codec x264` when cheap/old Android hardware is in scope. + +Details and citations: [`references/compatibility-matrix.md`](references/compatibility-matrix.md). + +## Quality policy + +- **Never upscale** — the source resolution is preserved; there is no `scale` in the pipeline. +- **HEVC** at `-cq 27`: measured **SSIM 0.9937** (visually transparent) at **~source size**. +- **H.264** (when chosen for reach) at `-crf 20`. +- **Audio** copied when already AAC; otherwise AAC 256k, where the residual error is the + source's own lossy error — 320k buys nothing measurable. +- `-fps_mode cfr` on every re-encode: without it the encoder silently drops 0.5–1.3% of + frames, leaving doubled intervals that play as judder. + +## Running it unattended + +Safe to schedule — it is idempotent and self-locking: + +- a **lock file** refuses a second instance (two concurrent NVENC jobs starve one encoder and + *drop frames*, the exact defect this tool removes) +- an already-playable file is skipped, so a re-run costs one probe +- a failed item leaves the input untouched, so the next run retries it +- aborts below 5 GB free rather than filling the work drive + +```sh +python scripts/transcode.py "Z:/media/incoming" --recursive \ + --out "Z:/media/converted" >> /var/log/transcode.log 2>&1 +``` + +## Install as a Hermes skill + +This repo is the canonical source. Copy it into the skill directory rather than editing the +skill copy in place: + +```sh +cp -r . "$LOCALAPPDATA/hermes/skills/media/universal-video-transcode" +``` + +See [`AGENTS.md`](AGENTS.md) for the non-negotiables. + +## Options + +| flag | meaning | +|---|---| +| `--dry-run` | print the plan, encode nothing | +| `--codec auto\|hevc\|x264` | force the video codec (`x264` for maximum reach) | +| `--cq N` / `--crf N` | quality (HEVC default 27 / H.264 default 20) | +| `--audio 256k`, `--channels 0` | audio bitrate; `0` keeps the source layout | +| `--cpu` | force libx265/libx264 (no NVENC) | +| `--keep` | keep the input file | +| `--overwrite` | replace an existing output | +| `--no-decode-check` | skip the full decode pass (faster, less safe) | +| `--lock PATH` | lock file location | + +`ffmpeg`/`ffprobe` are taken from `PATH`, or from `TRANSCODE_FFMPEG` / `TRANSCODE_FFPROBE`. diff --git a/SKILL.md b/SKILL.md new file mode 100644 index 0000000..720e6c4 --- /dev/null +++ b/SKILL.md @@ -0,0 +1,173 @@ +--- +name: universal-video-transcode +description: Use when transcoding video for iPad+Android playback. +version: 1.0.0 +author: Hermes Agent +license: MIT +platforms: [linux, windows] +metadata: + hermes: + tags: [ffmpeg, hevc, h264, mp4, ipad, ios, android, transcode, nvenc, cfr, frame-timing] + related_skills: [tubesync-selfhosted, local-audio-to-text] +--- + +# Universal video transcode — anything in, one MP4 that plays on iPad AND Android + +One tool: `scripts/transcode.py`. Hand it a **path**; a file converts, a folder converts +everything in it. It reads the source, picks the recipe itself, encodes, and verifies the +result before accepting it. + +```sh +python scripts/transcode.py "D:/video/clip.mkv" +python scripts/transcode.py "D:/video" --recursive --dry-run # plan only +python scripts/transcode.py "D:/video" --out D:/converted +``` + +## What "playable everywhere" actually means + +The intersection that both platforms decode **without a third-party player**: + +| | iPad / iOS | Android | +|---|---|---| +| Container | MP4/MOV only — **no Matroska** | MP4 and MKV both fine | +| Video | H.264 (all), HEVC `hvc1` (A9+) | H.264 Baseline mandatory; HEVC 5.0+ | +| Audio | AAC — **never Opus** | AAC and Opus both fine | + +**Apple is the constraint, not Android.** iOS has no Matroska demuxer and cannot play Opus +at all, so MP4 + AAC is what forces the conversion. Android would have accepted the `.mkv`. + +Two facts that decide the video codec: + +- **HEVC must be tagged `hvc1`.** ffmpeg defaults to `hev1`, which Apple refuses. `-tag:v hvc1` + is not optional. +- **Android only guarantees HEVC *Main profile Level 3*** (~SD). 1080p output is Level 4.0 + and 4K is 5.0 — fine on modern phones, but a low-end/old device may only reach Level 3. + When the audience includes cheap Android hardware, prefer H.264 (`--codec x264`), which is + mandatory from Android 3.0 and universal on iOS. + +Full matrix, per-platform citations, and the HDR/10-bit caveats: **`references/compatibility-matrix.md`**. + +## The three recipes it picks automatically + +Decided from the **source file**, never from its filename or extension: + +| recipe | when | video | audio | +|---|---|---|---| +| `copy` | H.264/HEVC with a usable timeline | `-c copy` — **bit-identical** | copy if AAC | +| `fix-audio` | same, audio is not AAC | `-c copy` | AAC 256k | +| `encode` | VP9/AV1, or H.264/HEVC whose timeline is unusable | HEVC (or x264) | copy if AAC, else AAC 256k | + +`copy` is used whenever a copy is possible, because a copy is lossless **by definition** — +that is the strongest possible answer to "keep the quality the same". + +## Quality policy ("as close to the original as possible") + +- **Never upscale.** The source resolution is preserved; there is no `scale` in the pipeline. +- **HEVC** targets `-cq 27`, measured on this hardware at **SSIM 0.9937** (visually transparent) + while landing at **~source size**. `-cq 28-30` buys 14-32% smaller at SSIM still >=0.992. +- **H.264** (when chosen for reach) targets `-crf 20`. +- **Audio**: copied when already AAC (bit-identical); otherwise **AAC 256k**, where the + residual error is the *source's own* lossy error — 320k buys nothing measurable. +- **Bitrate cap**: `-maxrate` = 1.5x the source bitrate, `-bufsize` = 2.5x that cap. Never set + `bufsize` equal to `maxrate` (a 1-second VBV buffer starves the encoder and collapses quality + on complex scenes); never cap at exactly the source bitrate (that pushes quality *below* the + original — a second-generation loss). + +## The two flags that must never be dropped + +```sh +-fps_mode cfr # without it the encoder silently drops 0.5-1.3% of frames, + # leaving doubled frame intervals that play as judder +-pix_fmt yuv420p # 8-bit 4:2:0; 10-bit / 4:2:2 / 4:4:4 break device compatibility +``` + +`-fps_mode cfr` makes ffmpeg *duplicate* a frame it cannot place instead of dropping it. +Duplicated frames cost almost nothing; a gap is visible. The tool adds both automatically and +fails the item if the output comes back non-uniform. + +## Verifying "no dropped frames" — and whether a browser is needed + +**Short answer: the browser is the arbiter, but it is NOT the gate. Automate the causal +check; use the browser only to spot-check.** + +The user-visible symptom (choppy playback) comes from a *deterministic* property of the +container: whether the frame timestamps are in **presentation** order or **decode** order. +That is readable from the header, so it needs no playback at all. + +The defect this matters for: a B-frame stream whose MP4 composition-offset table is +unusable — either a single constant `ctts` entry (`PTS == DTS + const`) or **no `ctts` box +at all** (`PTS == DTS`). The decoder still reorders by POC, so the timestamps attached to the +output frames are wrong and the compositor drops **~23%** of them — while +`r_frame_rate == avg_frame_rate` stays **EXACTLY** equal, so every fps-based test calls the +file healthy. + +| gate | cost | covers HEVC? | role | +|---|---|---|---| +| `r_frame_rate` vs `avg_frame_rate` | ~50 ms | yes | catches dropped/irregular cadence only | +| **`ctts` / PTS-inversion structural check** | ~50 ms | **yes** | **the automatable gate — causal** | +| full decode (`ffmpeg -f null -`) | minutes | yes | decode integrity | +| real Chrome `droppedVideoFrames` | ~30 s/file | **NO** | arbiter for H.264 spot-checks | + +Three traps that make the browser check unsuitable as an automated gate: + +1. **Chrome has no HEVC decoder.** `canPlayType('video/mp4; codecs="hvc1…"')` returns `''`, so an + HEVC file reports **0 decoded / 0 dropped** and *looks perfect while nothing played*. A + 0-frame result is **not** a pass. HEVC can only be confirmed on the Apple device itself. +2. It needs a real Chrome and a real page; ~30 s per file is far too slow for a library. +3. The measurement is statistical (a 1-frame drop is noise), so it needs a threshold, not equality. + +The tool therefore runs the structural gate on **every** file automatically, plus a full decode +pass, and refuses the output otherwise. Measurements, the exact thresholds, and the +2x2 experiment that proved causality: **`references/verification-and-automation.md`**. + +## Running it unattended (cron) + +Safe to schedule, because it is idempotent and self-locking: + +- **A lock file** (default `~/.transcode.lock`) refuses to start a second instance. This is not + optional hygiene: two concurrent NVENC jobs share one encoder unit, starve each other, and + **drop frames** — the exact defect the tool exists to prevent. +- **Idempotent**: a file that is already a playable MP4 is skipped, so a re-run costs a probe + and nothing else. `--dry-run` prints the plan without encoding. +- **A failed item leaves the input untouched**, so the next run simply retries it. +- **Disk guard**: aborts below 5 GB free rather than filling the work drive (an ENOSPC + mid-encode is reported as a misleading codec error and costs every remaining item). + +```sh +# hourly; exits non-zero if anything failed +python /path/to/scripts/transcode.py "Z:/docker/tubesync/download" --recursive \ + --out "Z:/docker/tubesync/converted" >> /var/log/transcode.log 2>&1 +``` + +For a TubeSync library specifically, prefer its own DB-driven pipeline (`tubesync-selfhosted`) +— it also updates the database rows. Use this tool for anything outside that library. + +## Pitfalls + +- **Never write the encode output directly to a network share.** A streaming `.part` write over + SMB/SFTP/Dokan leaves a corrupt stub when the connection blips. Encode to local disk, verify, + then copy the complete file across. The tool writes `dst + ".part.mp4"` and renames only after + verification — keep it on a local path if the destination is remote. +- **`os.path.getsize()` lies on RaiDrive/Dokan** — it returns cached metadata from the *previous* + file handle. Verify a network copy by sequential chunked read, never by size. +- **A `.mkv` extension on a converted file is cosmetic.** TubeSync's rename pass rewrites + finished `.mp4` files back to `.mkv` while `downloaded_container` still reads `mp4`, and + `Content-Type` follows the container. Do not "fix" it and do not report it as a playback bug. +- **Do not re-download to fix a timing defect.** The mkv that yt-dlp writes is itself flat, so a + re-download reproduces the defect exactly. +- **A `-c copy` remux cannot repair timing.** ffmpeg will not rewrite `stts`/`ctts` under stream + copy, and retiming only the SPS (`-bsf:v h264_metadata=tick_rate=…`) changes nothing either. + Repairing a copy-tier file means re-encoding its video — quote the size cost (1.28x at crf 23, + 1.74x at crf 20) before applying it to a whole library. +- **Unicode paths**: on Windows, drive ffmpeg from Python `subprocess` with an argv list. MSYS + bash cannot hand a Chinese filename to native `ffmpeg.exe` even when `ls` shows it. +- **`-hwaccel cuda` alone may still software-decode VP9**; `-c:v vp9_cuvid` forces NVDEC. + +## Support files + +- `scripts/transcode.py` — the tool. `--dry-run`, `--codec`, `--cq/--crf`, `--audio`, + `--channels`, `--cpu`, `--keep`, `--overwrite`, `--no-decode-check`, `--lock`. +- `references/compatibility-matrix.md` — per-platform codec/container facts with citations, + the HEVC level caveat, and what breaks on each device. +- `references/verification-and-automation.md` — the defect classes, the structural gate, the + browser arbiter and its blind spot, and the 2x2 causality proof. diff --git a/references/compatibility-matrix.md b/references/compatibility-matrix.md new file mode 100644 index 0000000..10c0247 --- /dev/null +++ b/references/compatibility-matrix.md @@ -0,0 +1,95 @@ +# Compatibility matrix — what actually plays where + +Sources: Android's [Supported media formats](https://developer.android.com/media/platform/supported-formats) +(read 2026-10-04) and Apple's HEVC hardware baseline (A9, announced 2015-09-09 with the +iPhone 6S — its first chip with a hardware HEVC decoder, **Main 8 and Main 10**). + +## The intersection + +The target is the set that plays on **both** platforms with **no third-party player**: + +```text +Container : MP4 (ISO BMFF) +Video : H.264 High 8-bit yuv420p ── universal ── + or HEVC Main 8-bit yuv420p, tagged hvc1 ── smaller ── +Audio : AAC-LC +Extras : moov before mdat (+faststart), uniform frame cadence +``` + +**Apple is the constraint, not Android.** Android decodes Matroska and Opus natively; iOS has +no Matroska demuxer and cannot play Opus at all. So `.mkv` + Opus → MP4 + AAC is the change +that forces the whole conversion. + +## Android (from the official table) + +| Format | Decoder | Containers | Notes | +|---|---|---|---| +| H.264 AVC **Baseline** | **YES** (all versions) | 3GP, MP4, MPEG-TS | the only unconditional guarantee | +| H.264 AVC **Main** | **YES** | — | "The decoder is required, the encoder is recommended"; encoder Main is Android 6.0+ | +| **H.265 HEVC** | **Android 5.0+** | MP4, Matroska | **"Main Profile Level 3 for mobile devices and Main Profile Level 4.1 for Android TV"** | +| VP8 | Android 2.3.3+ | WebM, Matroska | | +| VP9 | Android 4.4+ | WebM, Matroska, MP4 | | +| AV1 | Android 10+ (mandatory 14+) | MP4, Matroska | | +| AAC-LC | YES | 3GP, MP4, m4a, ADTS | mono/stereo/5.0/5.1, 8–48 kHz | +| Opus | Android 5.0+ | Ogg, MP4, Matroska | irrelevant for iOS targets | + +### The HEVC level caveat — read this before promising "HEVC plays on Android" + +Android's guarantee is **Main profile Level 3**, which is roughly SD. Our 1080p output declares +**Level 4.0** and 4K declares **5.0** (measured). Modern phones decode far beyond that, but a +low-end or old device is only *required* to reach Level 3 — so "Android supports HEVC" does not +imply "Android supports *this* HEVC". + +- Audience includes cheap/old Android hardware → prefer **H.264** (`--codec x264`). +- Modern devices only → HEVC is fine and buys a large size reduction. + +## Apple / iOS + +| Format | Support | +|---|---| +| Container | **MP4 / MOV only.** No Matroska demuxer in Photos, Files, TV or Safari. | +| H.264 | All devices. | +| HEVC | **A9 and newer** (iPhone 6S / 2015 iPad and later). Main 8 and Main 10. | +| `hvc1` vs `hev1` | **`hvc1` required.** Apple refuses `hev1`; ffmpeg defaults to `hev1`. | +| VP9 / VP09 | **Never.** | +| AV1 | Only A17 Pro / M3 and newer. | +| Opus | **Never.** | +| AAC | Yes. | + +`-tag:v hvc1` is the single most commonly missed flag: a file that is otherwise perfect will be +refused by an iPad purely because it carries the `hev1` sample entry. + +## Pixel format and bit depth + +| pix_fmt | Verdict | +|---|---| +| **`yuv420p`** | **use this** — 8-bit 4:2:0, universal on both platforms | +| `yuv420p10le` | Apple accepts it (A9+, Main 10); Android's guarantee is Main profile, so treat as a compatibility risk. Not needed for SDR sources. | +| `yuv422p` / `yuv444p` | **avoid** — outside the guaranteed profile for both | + +**HDR is a separate question.** All VP9/VP09/AV1 sources sampled from this library were +**Profile 0 = 8-bit, yuv420p, bt709**; HDR was 1 file in 4431. For SDR sources there is nothing +to preserve, and tone-mapping is out of scope for the tool. + +## Container and streaming details + +- `moov` **before** `mdat` (`-movflags +faststart`) so playback starts without downloading the + whole file. Verify: the first 32 bytes are `…ftyp isom`, then `moov`, then `mdat`. +- **Range requests must work** — the player seeks with them. Expect `206` + + `Content-Range: bytes …` + `Accept-Ranges: bytes`. +- `Content-Type` should be `video/mp4`. A converted file served as `video/matroska` can be + refused by AVPlayer even though the bytes are a valid MP4. +- **The filename extension is not authoritative.** A `.mkv` name on an MP4 body streams fine + when `Content-Type` follows the container. Do not "fix" the extension; do fix the MIME. + +## Audio decisions + +| source audio | action | result | +|---|---|---| +| AAC | **copy** (`-c:a copy`) | bit-identical to the original | +| Opus / Vorbis / anything else | **AAC 256k** | required — Opus is invalid in MP4 and iOS cannot play it | + +Measured cascade error (source Opus decode vs AAC decode, sample-aligned): 128k = 22.7 dB, +192k = 29.0 dB, **256k = 33.5 dB**, 320k = 36.1 dB. Above 256k the residual is the source's own +Opus loss, so 320k buys almost nothing. Cost of 256k over a 626-hour library: +0.5 MB/min +≈ +19 GB (~3%), which is why audio stops being a concern at 256k. diff --git a/references/verification-and-automation.md b/references/verification-and-automation.md new file mode 100644 index 0000000..468eb60 --- /dev/null +++ b/references/verification-and-automation.md @@ -0,0 +1,124 @@ +# Verifying "it plays smoothly" — the gate, the arbiter, and what is automatable + +## The question this answers + +"Is there a dropped frame?" and "do I have to play it in a browser to know?" — no. The +browser is the *arbiter* of the user-visible symptom; it is not the *gate*. Automate the +causal check, spot-check with the browser. + +The symptom comes from a deterministic property of the container, readable from the header: +whether the frame timestamps are in **presentation** order or **decode** order. + +## The defect classes + +Three distinct causes produce the same complaint ("一卡一卡 / choppy"), and a file can have +any combination. Only the first is visible to an fps test. + +| # | class | signature | fps test sees it? | +|---|---|---|---| +| 1 | non-uniform cadence | `r_frame_rate` != `avg_frame_rate` (gap > 0.01) | **yes** | +| 2 | encoder dropped frames | doubled intervals (0.040 and 0.080 where 0.040 belongs) | yes (avg drops) | +| 3 | **unusable composition offsets** | `r == avg` **exactly** | **NO** | + +Class 3 is the one that hides. Two shapes, both the same defect: + +- MP4 with a **single constant `ctts` entry** → `PTS == DTS + const` +- MP4 with **no `ctts` box at all** → `PTS == DTS` + +Either way the timestamps are in decode order. Decoders still reorder by POC, so the +timestamps attached to the *output* frames are wrong, and the compositor drops ~23% of them. + +> **A missing `ctts` is only a defect on an MP4 whose video stream reorders.** Matroska stores +> presentation timestamps per block and legitimately has no `ctts` box — so a detector that +> treats "no ctts" as flat will flag every `.mkv`. Distinguish "no MP4 sample table" from +> "MP4 with no ctts box", and use the packet-PTS test for Matroska. + +## The gate (automatable, runs on every file) + +Both checks are header-only (~50 ms) and **work on HEVC**, unlike the browser. + +**MP4 — count the `ctts` entries.** Walk the `moov` from the first 2 MB (the `moov` is at the +front after `+faststart`). `<= 1` entry on a stream with `has_b_frames > 0` is the defect. +Healthy files in this library carry **1,500–43,000** entries. + +**Any container — count PTS inversions.** A stream with B-frames *must* present frames out of +decode order, so its container PTS go backwards somewhere: + +```sh +ffprobe -v error -select_streams v:0 -read_intervals "%+3" \ + -show_entries packet=pts -of csv=p=0 FILE +``` + +Count steps where `pts[i+1] < pts[i]`. **0 inversions = flat/broken**; 20–45 = healthy. +This is the only option for Matroska, which has no `ctts`. + +Measured separation, no overlap: + +| file | ctts entries | PTS inversions | Chrome dropped | +|---|---|---|---| +| defective | **1** | **0** | **22.8–27.3%** | +| defective (no ctts box) | **absent** | **0** | **22.8–23.5%** | +| same file, x264 CFR re-encode | 985 | 35 | **0.0%** | +| known-good 1080p | 1549 | 23 | 0.0% | +| known-good 360p | 4262 | 35 | 0.2% | +| our HEVC 2160p output | many | 44 | (not measurable in Chrome) | + +A file that is *already* a playable MP4 must also be checked before being skipped, or a +"nothing to do" verdict is issued for a file that stutters. + +## The arbiter (real Chrome) and its blind spot + +```sh +python measure_playback.py [seconds] # add a second url to A/B in one run +``` + +Plays muted in real headless Chrome over CDP and reports `getVideoPlaybackQuality()` — +decoded frames, `droppedVideoFrames`, and the media-time/wall-clock ratio. + +- **The ratio is not the verdict.** A defective file advances at ratio **1.000** while dropping + 20–23% of frames. Only the drop count separates healthy from broken. +- **Threshold, not equality.** A 1–2 frame drop at startup is noise (healthy files measure + 0.0–0.2%); a real defect measures ~20%+. Treat `< 1%` as smooth. +- **Chrome cannot decode HEVC.** `canPlayType('video/mp4; codecs="hvc1…"')` returns `''`, so an + HEVC file reports **0 decoded / 0 dropped** and looks perfect while nothing played. A 0-frame + result is **not a pass** — it means "not measurable here". Judge HEVC by the structural gate + and confirm on the Apple device. + +So the browser is: too slow (~30 s/file), unable to judge the main output codec, and +statistical. It is the right tool for an H.264 A/B spot-check, not for a library gate. + +## Why this is causal, not a heuristic — the 2x2 + +Correlation would not be enough (a defect could be a symptom of something else). The 2x2 +crosses the container timing against the bitstream: + +| | flat container | correct offsets | +|---|---|---| +| **healthy bitstream** | **23.5% dropped** (stutter) | 0.0% | +| **defective bitstream** | 22.8% dropped | **0.0%** (fixed) | + +Writing the same flat container around a *healthy* bitstream reproduces the stutter, and +writing proper offsets around the *broken* bitstream fixes it. The bitstream is irrelevant — +**the container timing is the cause**. That is what licenses a header-only gate. + +## What cannot fix it + +- `-c copy` cannot retime. ffmpeg will not rewrite `stts`/`ctts` under stream copy; output + timing stays byte-identical and the drop count is unchanged. +- Repacking the raw H.264 (`-fflags +genpts -r 25 -c copy`) stays flat — measured. +- `-bsf:v h264_metadata=tick_rate=…` retimes only the SPS; timestamps unchanged. +- Re-downloading reproduces it: the `.mkv` yt-dlp writes is itself flat. + +Only a real video re-encode regenerates the timeline. Cost on a copy-tier file: 1.28x at +crf 23, 1.74x at crf 20, 2.17x at crf 18. + +## Provenance: inherited, not introduced by the remux + +The defect arrives with the download. Some `.mkv` files yt-dlp writes are already flat (PTS +monotonic in file order, `dts = pts - 40`), and `-c copy` carries that through unchanged. An +mp4→mkv→mp4 round trip does not repair it. + +Measured on one channel folder: **8 of 8** `360p-avc1` source `.mkv` files were flat, and the +same key was flat in both its `.mkv` and its converted `.mp4`. It is source-specific, not +library-wide: ~1.3% of AVC1 files, and **0 of 60** in the re-encoded HEVC tier (a re-encode +regenerates timing, so the HEVC path is structurally immune). diff --git a/scripts/selftest.py b/scripts/selftest.py new file mode 100644 index 0000000..c9786cd --- /dev/null +++ b/scripts/selftest.py @@ -0,0 +1,174 @@ +#!/usr/bin/env python3 +"""selftest - prove the frame-timing detector actually works, on this machine. + +The whole value of this repo rests on one claim: a file can drop ~23% of its frames in a +browser while `r_frame_rate == avg_frame_rate` EXACTLY, and the defect is decidable from +the header alone. So the detector is tested against files built here, with both defect +shapes and two must-not-false-positive cases: + + src.mp4 healthy MP4, B-frames, proper ctts -> must NOT be flagged + src.mkv same stream in Matroska (no ctts by design) -> must NOT be flagged + flat_noctts.mp4 MP4 video track with NO ctts box -> MUST be flagged + flat_ctts1.mp4 MP4 with a single constant ctts entry -> MUST be flagged + +`flat_ctts1.mp4` is produced by binary-patching the real ctts box down to one entry and +padding the difference with a `free` box, so the moov keeps its exact size and no chunk +offsets shift. That is the shape YouTube/yt-dlp actually produced in the field. + +Exit code 0 = all assertions held. Non-zero = the detector is wrong, and the tool cannot +be trusted to gate conversions. + +Usage: python scripts/selftest.py +""" +import os +import shutil +import struct +import subprocess +import sys +import tempfile + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +import transcode as T # noqa: E402 + +FFMPEG = T.FFMPEG +FFPROBE = T.FFPROBE + +NESTED = (b"moov", b"trak", b"mdia", b"minf", b"stbl") + + +def run(cmd): + return subprocess.run(cmd, capture_output=True, text=True, + encoding="utf-8", errors="replace") + + +def find_box(buf, typ): + """Walk the (nested) box tree, return (offset, size) of the first `typ`, else None.""" + def walk(start, end): + p = start + while p + 8 <= end: + size = struct.unpack(">I", buf[p:p + 4])[0] + name = buf[p + 4:p + 8] + hdr = 8 + if size == 1: + size = struct.unpack(">Q", buf[p + 8:p + 16])[0] + hdr = 16 + elif size == 0: + size = end - p + if size < 8 or p + size > end: + return None + if name == typ: + return (p, size) + if name in NESTED: + got = walk(p + hdr, p + size) + if got: + return got + p += size + return None + return walk(0, len(buf)) + + +def make_flat_ctts1(src, dst): + """Rewrite the video track's ctts to ONE constant entry, same box size. + + A ctts entry is (sample_count, sample_offset); one entry covering every sample with a + constant offset is exactly the defect: PTS == DTS + const for the whole stream. + The byte difference is absorbed by a `free` box so the moov size is unchanged and the + chunk offset tables stay valid. + """ + buf = bytearray(open(src, "rb").read()) + hit = find_box(bytes(buf), b"ctts") + if not hit: + raise SystemExit("selftest: no ctts in %s - cannot build the flat_ctts1 fixture" % src) + off, size = hit + ver = buf[off + 8] + n = struct.unpack(">I", buf[off + 12:off + 16])[0] + total, first_off = 0, 0 + for i in range(n): + c, o = struct.unpack(">Ii" if ver == 1 else ">II", buf[off + 16 + 8 * i:off + 24 + 8 * i]) + if i == 0: + first_off = o + total += c + new = struct.pack(">I", 24) + b"ctts" + bytes([ver, 0, 0, 0]) + \ + struct.pack(">I", 1) + struct.pack(">Ii" if ver == 1 else ">II", total, first_off) + pad = size - len(new) + if pad < 8: + raise SystemExit("selftest: ctts too small to patch in place (%d bytes)" % size) + filler = struct.pack(">I", pad) + b"free" + b"\x00" * (pad - 8) + buf[off:off + size] = new + filler + open(dst, "wb").write(bytes(buf)) + return total + + +def main(): + if not (shutil.which(FFMPEG) or os.path.exists(FFMPEG)): + raise SystemExit("ffmpeg not found - set TRANSCODE_FFMPEG or put ffmpeg on PATH") + d = tempfile.mkdtemp(prefix="tctest-") + print("work dir: %s\n" % d) + try: + src = os.path.join(d, "src.mp4") + # B-frames are required: without reordering there is nothing to get wrong, and the + # detector is correct to stay silent. + r = run([FFMPEG, "-v", "error", "-y", "-f", "lavfi", + "-i", "testsrc=size=320x240:rate=25", "-t", "3", + "-c:v", "libx264", "-preset", "ultrafast", "-bf", "3", + "-pix_fmt", "yuv420p", src]) + if r.returncode != 0: + raise SystemExit("ffmpeg could not build the source: %s" % r.stderr) + + hbf = run([FFPROBE, "-v", "error", "-select_streams", "v:0", + "-show_entries", "stream=has_b_frames", "-of", "csv=p=0", src]).stdout.strip() + if hbf in ("", "0"): + raise SystemExit("fixture has no B-frames (has_b_frames=%r) - test is meaningless" % hbf) + + raw = os.path.join(d, "raw.h264") + flat_no = os.path.join(d, "flat_noctts.mp4") + run([FFMPEG, "-v", "error", "-y", "-i", src, "-c", "copy", + "-bsf:v", "h264_mp4toannexb", "-f", "h264", raw]) + run([FFMPEG, "-v", "error", "-y", "-fflags", "+genpts", "-r", "25", + "-i", raw, "-c", "copy", flat_no]) + + flat_c1 = os.path.join(d, "flat_ctts1.mp4") + total = make_flat_ctts1(src, flat_c1) + + mkv = os.path.join(d, "src.mkv") + run([FFMPEG, "-v", "error", "-y", "-i", src, "-c", "copy", mkv]) + + cases = [ + (src, False, "healthy MP4, proper ctts"), + (mkv, False, "Matroska - no ctts by design, must NOT be flagged"), + (flat_no, True, "MP4 with NO ctts box (PTS == DTS)"), + (flat_c1, True, "MP4 with a single constant ctts entry (PTS == DTS + const)"), + ] + + print("ctts table: %d samples folded into one constant entry\n" % total) + print("%-34s %-8s %-8s %s" % ("fixture", "expect", "got", "verdict")) + bad = 0 + for path, expect, note in cases: + info = T.probe(path) + got = T.flat_timing(info, path) + ok = (got == expect) + bad += 0 if ok else 1 + print("%-34s %-8s %-8s %-5s %s" + % (os.path.basename(path), expect, got, "ok" if ok else "FAIL", note)) + + # The detector must also drive the right DECISION, not just a boolean: a flat file + # cannot be fixed by -c copy, so plan_for has to choose a real re-encode. + print() + for path, want_encode, note in ((src, False, "healthy -> copy is safe"), + (flat_c1, True, "flat -> must re-encode")): + vmode, _, why = T.plan_for(T.probe(path), path, "auto") + must_reencode = vmode in ("x264", "hevc") + ok = (must_reencode == want_encode) + bad += 0 if ok else 1 + print("%-34s plan=%-6s %-5s %s" + % (os.path.basename(path), vmode, "ok" if ok else "FAIL", note)) + print("%-34s reason: %s" % ("", why)) + + print("\n%s" % ("ALL ASSERTIONS HELD" if not bad else "%d ASSERTION(S) FAILED" % bad)) + return 1 if bad else 0 + finally: + shutil.rmtree(d, ignore_errors=True) + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/transcode.py b/scripts/transcode.py new file mode 100644 index 0000000..42bcf58 --- /dev/null +++ b/scripts/transcode.py @@ -0,0 +1,492 @@ +#!/usr/bin/env python3 +"""transcode - turn ANY video/audio file into an MP4 that plays on BOTH iPad and Android, +keeping quality as close to the source as the target codec allows. + +Give it a path. A file converts; a folder converts everything in it. Nothing else is needed. + + python transcode.py "D:/video/clip.mkv" + python transcode.py "D:/video" --recursive + python transcode.py "D:/video" --dry-run # plan only, no encoding + +Three recipes are chosen automatically from the SOURCE FILE (never from a filename): + + copy-video source is H.264/HEVC with a usable timeline -> remux to MP4, bit-identical + fix-audio same, but the audio is not AAC -> copy video, re-encode audio to AAC + encode VP9/AV1, or H.264 whose timeline is unusable -> full HEVC (or x264) re-encode + +Quality policy ("as close to the original as possible"): + * never upscale; the source resolution is preserved + * video is COPIED whenever a copy is possible - that is lossless by definition + * HEVC re-encode targets -cq 27, measured at SSIM 0.9937 == visually transparent, + landing at ~source size. x264 (for browser-only targets) targets -crf 20. + * audio is COPIED when it is already AAC (bit-identical); otherwise AAC 256k, where the + residual error is the source's own lossy error, so more bits buy nothing. + +Every output is verified before it is accepted; a failed item leaves the input untouched. +""" +import argparse +import json +import os +import shutil +import subprocess +import sys +import time + +FFMPEG = os.environ.get("TRANSCODE_FFMPEG", "ffmpeg") +FFPROBE = os.environ.get("TRANSCODE_FFPROBE", "ffprobe") +MIN_FREE_GB = 5 +VIDEO_EXT = {".mkv", ".mp4", ".mov", ".avi", ".webm", ".flv", ".ts", ".m4v", ".wmv", + ".mpg", ".mpeg", ".m2ts", ".3gp", ".ogv", ".vob"} +AUDIO_EXT = {".m4a", ".aac", ".mp3", ".opus", ".flac", ".wav", ".ogg", ".wma"} +SKIP_EXT = {".jpg", ".jpeg", ".png", ".nfo", ".vtt", ".srt", ".json", ".txt", ".part", + ".ytdl", ".tmp"} + + +def log(*a): + print(time.strftime("%H:%M:%S ") + " ".join(str(x) for x in a), flush=True) + + +def run(cmd, timeout=86400): + return subprocess.run(cmd, capture_output=True, text=True, + encoding="utf-8", errors="replace", timeout=timeout) + + +def _fps(v): + try: + n, d = (v or "0/0").split("/") + return float(n) / float(d) if float(d) else 0.0 + except Exception: + return 0.0 + + +def probe(path): + r = run([FFPROBE, "-v", "error", "-print_format", "json", + "-show_streams", "-show_format", path], timeout=120) + try: + j = json.loads(r.stdout) + except Exception: + return None + v = next((s for s in j.get("streams", []) if s.get("codec_type") == "video"), None) + a = next((s for s in j.get("streams", []) if s.get("codec_type") == "audio"), None) + if not v: + return None + return { + "vcodec": (v.get("codec_name") or "").lower(), + "tag": (v.get("codec_tag_string") or "").lower(), + "w": v.get("width"), "h": v.get("height"), + "pix": (v.get("pix_fmt") or "").lower(), + "acodec": (a.get("codec_name") or "").lower() if a else "", + "channels": a.get("channels") if a else 0, + "dur": float(j.get("format", {}).get("duration") or 0), + "size": int(j.get("format", {}).get("size") or 0), + "fmt": (j.get("format", {}).get("format_name") or "").lower(), + "rfps": _fps(v.get("r_frame_rate")), "afps": _fps(v.get("avg_frame_rate")), + "hbf": int(v.get("has_b_frames") or 0), + } + + +def _ctts_entries(path): + """ctts entry count from the moov; -1 if the MP4 video track has NO ctts box at all; + None if there is no MP4 sample table (not an MP4, or unreadable). + + The -1/None distinction matters: a missing ctts is a DEFECT on an MP4 with B-frames, + but is perfectly normal in Matroska, which stores presentation timestamps per block. + """ + try: + with open(path, "rb") as f: + head = f.read(2_000_000) + except Exception: + return None + p, saw_stbl = 0, False + while p + 8 <= len(head): + size = int.from_bytes(head[p:p + 4], "big") + typ = head[p + 4:p + 8] + hdr = 8 + if size == 1: + size = int.from_bytes(head[p + 8:p + 16], "big"); hdr = 16 + elif size == 0: + break + if size < 8: + break + if typ == b"ctts": + return int.from_bytes(head[p + 12:p + 16], "big") + if typ == b"stbl": + saw_stbl = True + if typ in (b"moov", b"trak", b"mdia", b"minf", b"stbl"): + p += hdr + continue + p += size + return -1 if saw_stbl else None + + +def flat_timing(info, path): + """True when a B-frame MP4 track carries no usable composition offsets. + + Either a single constant ctts entry (PTS == DTS + const) or no ctts box at all + (PTS == DTS). Both put the frame timestamps in DECODE order; the decoder still + reorders by POC, so the timestamps on the output frames are wrong and the compositor + drops ~23% of them. r_frame_rate == avg_frame_rate EXACTLY, so no fps test sees it. + """ + if not info or info["hbf"] <= 0: + return False + ce = _ctts_entries(path) + if ce is None: + return False # not an MP4 video track - not our call to make + return ce <= 1 + + +def pts_inversions(path, seconds=3): + """(inversions, packets) over the first `seconds` of video packets. + + A stream with B-frames MUST present frames out of decode order, so its container PTS + go backwards somewhere: 0 inversions means the timestamps are in decode order. + """ + r = run([FFPROBE, "-v", "error", "-select_streams", "v:0", + "-read_intervals", "%%+%d" % seconds, + "-show_entries", "packet=pts", "-of", "csv=p=0", path], timeout=180) + vals = [] + for line in (r.stdout or "").splitlines(): + t = line.strip().split(",")[0] + if t.lstrip("-").isdigit(): + vals.append(int(t)) + return sum(1 for i in range(len(vals) - 1) if vals[i + 1] < vals[i]), len(vals) + + +def unusable_timeline(info, path): + """(bool, why) - frame timestamps that no player can present smoothly. + + Covers BOTH defect shapes in one gate, across containers: + * MP4 -> ctts missing, or a single constant entry (PTS == DTS + const) + * Matroska -> no ctts by design, so read the packet PTS instead; 0 inversions on a + B-frame stream means decode-order timestamps + Either way the decoder reorders by POC and the compositor drops ~23% of frames, while + r_frame_rate == avg_frame_rate stays EXACTLY equal, so no fps test can see it. + """ + if not info or info["hbf"] <= 0: + return False, "" + ce = _ctts_entries(path) + if ce is not None: + if ce <= 1: + return True, "flat composition offsets (ctts=%s)" % (ce if ce >= 0 else "missing") + return False, "" + inv, n = pts_inversions(path) + if n >= 20 and inv == 0: + return True, "flat timestamps in %s (0 PTS inversions over %d packets)" % (info["fmt"], n) + return False, "" + + +def has_nvenc(): + try: + r = run([FFMPEG, "-hide_banner", "-encoders"], timeout=60) + return "hevc_nvenc" in (r.stdout or "") + except Exception: + return False + + +def plan_for(info, path, force_codec): + """Return (video_mode, audio_mode, reason). video_mode: copy | x264 | hevc.""" + if info["dur"] < 1.0: + return None, None, "stub (needs re-download, not a transcode)" + if info["vcodec"] not in ("h264", "avc1", "hevc", "vp9", "vp09", "av1", "av01"): + return None, None, "unsupported video codec %s" % info["vcodec"] + + acopy = info["acodec"] in ("aac", "mp4a", "") + amode = "copy" if acopy else "aac" + + is_mp4 = ("mp4" in info["fmt"]) or ("mov" in info["fmt"]) + flat, why_flat = unusable_timeline(info, path) + uniform = abs(info["rfps"] - info["afps"]) <= 0.01 + + if info["vcodec"] in ("h264", "avc1"): + # A copy is lossless, but only if the timeline is already correct: ffmpeg cannot + # retime under -c copy, so a defective one would survive into the output. + if flat: + return "x264", amode, "H.264 but %s - copy cannot retime" % why_flat + if not uniform: + return "x264", amode, "H.264 but non-uniform cadence - copy cannot retime" + if not is_mp4: + return "copy", amode, "H.264 in %s -> lossless remux to MP4" % (info["fmt"] or "?") + if info["tag"] and info["tag"] != "avc1": + return "copy", amode, "re-tag %s -> avc1" % info["tag"] + if not acopy: + return "copy", amode, "H.264 + %s audio -> copy video, fix audio" % info["acodec"] + return None, None, "already playable MP4 (H.264 + AAC, correct timeline)" + + if info["vcodec"] == "hevc": + if is_mp4 and info["tag"] == "hvc1" and not flat and uniform and acopy: + return None, None, "already playable MP4 (HEVC hvc1 + AAC, correct timeline)" + if flat or not uniform: + return "hevc", amode, "HEVC but %s - must re-encode" % (why_flat or "non-uniform cadence") + return "copy", amode, "HEVC -> remux/tag hvc1 + fix audio" + + want = force_codec if force_codec != "auto" else ("x264" if info["vcodec"] == "h264" else "hevc") + return want, amode, "%s -> %s re-encode" % (info["vcodec"].upper(), want) + + +def build_cmd(src, dst, info, vmode, amode, args, use_nvenc): + cmd = [FFMPEG, "-y", "-hide_banner", "-loglevel", "warning", "-stats"] + if vmode == "hevc" and use_nvenc: + dec = {"vp9": "vp9_cuvid", "vp09": "vp9_cuvid", "av1": "av1_cuvid", + "av01": "av1_cuvid", "h264": "h264_cuvid", "avc1": "h264_cuvid", + "hevc": "hevc_cuvid"}.get(info["vcodec"]) + if dec: + cmd += ["-c:v", dec] + cmd += ["-i", src, "-map", "0:v:0", "-map", "0:a:0?", "-map_metadata", "0"] + + if vmode == "copy": + cmd += ["-c:v", "copy"] + elif vmode == "x264": + cmd += ["-c:v", "libx264", "-preset", args.preset, "-crf", str(args.crf), + "-profile:v", "high", "-level:v", "4.0", "-pix_fmt", "yuv420p"] + else: + if use_nvenc: + br = info["size"] * 8 / max(info["dur"], 1) + cmd += ["-c:v", "hevc_nvenc", "-preset", args.nvenc_preset, "-tune", "hq", + "-rc", "vbr", "-cq", str(args.cq), "-b:v", "0", + "-maxrate", str(int(br * 1.5)), "-bufsize", str(int(br * 1.5 * 2.5))] + else: + cmd += ["-c:v", "libx265", "-preset", args.preset, "-crf", str(args.cq)] + cmd += ["-profile:v", "main", "-pix_fmt", "yuv420p", "-tag:v", "hvc1"] + + # -fps_mode cfr is MANDATORY on every re-encode: without it the encoder silently drops + # the frames it cannot place (measured 0.5-1.3%), leaving doubled frame intervals that + # play as judder. Copy modes keep the source's own timing, which was verified above. + if vmode != "copy": + cmd += ["-fps_mode", "cfr"] + + if amode == "copy": + cmd += ["-c:a", "copy"] + else: + cmd += ["-c:a", "aac", "-b:a", args.audio] + if args.channels: + cmd += ["-ac", str(args.channels)] + + cmd += ["-movflags", "+faststart", dst] + return cmd + + +def verify(src_info, out, vmode, amode, check_decode=True): + """Structural gate. Returns (ok, message). Every claim is read back from the file.""" + oi = probe(out) + if not oi: + return False, "output unreadable" + if oi["dur"] < 1.0: + return False, "output is a stub" + if abs(oi["dur"] - src_info["dur"]) > max(1.0, src_info["dur"] * 0.01): + return False, "duration %.2f vs source %.2f" % (oi["dur"], src_info["dur"]) + if (oi["w"], oi["h"]) != (src_info["w"], src_info["h"]): + return False, "resolution %sx%s vs %sx%s" % (oi["w"], oi["h"], src_info["w"], src_info["h"]) + if vmode == "hevc": + if oi["vcodec"] != "hevc": + return False, "vcodec %s (want hevc)" % oi["vcodec"] + if oi["tag"] != "hvc1": + return False, "codec tag %s (Apple requires hvc1)" % oi["tag"] + if vmode == "x264" and oi["vcodec"] not in ("h264", "avc1"): + return False, "vcodec %s (want h264)" % oi["vcodec"] + if src_info["acodec"] and oi["acodec"] != "aac": + return False, "audio %s (want aac)" % oi["acodec"] + if oi["pix"] not in ("yuv420p", "yuvj420p"): + return False, "pixel format %s (want 8-bit 4:2:0)" % oi["pix"] + if vmode != "copy": + if abs(oi["rfps"] - oi["afps"]) > 0.01: + return False, "non-uniform cadence r=%.3f avg=%.3f (is -fps_mode cfr missing?)" % ( + oi["rfps"], oi["afps"]) + if flat_timing(oi, out): + return False, "flat composition offsets survived (ctts<=1 on a B-frame stream)" + if check_decode: + r = run([FFMPEG, "-v", "error", "-i", out, "-f", "null", "-"], timeout=7200) + if r.returncode != 0: + return False, "decode errors: %s" % (r.stderr or "").strip()[:200] + return True, "ok" + + +def out_path(src, args): + stem = os.path.splitext(os.path.basename(src))[0] + d = args.out or os.path.dirname(os.path.abspath(src)) + return os.path.join(d, stem + ".mp4") + + +class Lock: + """One encoder at a time. Two concurrent NVENC jobs starve each other and drop frames - + the exact defect this tool exists to prevent - so cron runs must never overlap.""" + + def __init__(self, path): + self.path = path + self.fd = None + + def __enter__(self): + os.makedirs(os.path.dirname(os.path.abspath(self.path)) or ".", exist_ok=True) + try: + self.fd = os.open(self.path, os.O_CREAT | os.O_EXCL | os.O_WRONLY) + os.write(self.fd, str(os.getpid()).encode()) + except FileExistsError: + try: + pid = int(open(self.path).read().strip() or 0) + except Exception: + pid = 0 + if pid and _alive(pid): + raise SystemExit("another transcode is already running (pid %d, %s) - refusing " + "to start a second encoder" % (pid, self.path)) + log("stale lock (pid %s gone) - taking it over" % pid) + os.remove(self.path) + self.fd = os.open(self.path, os.O_CREAT | os.O_EXCL | os.O_WRONLY) + os.write(self.fd, str(os.getpid()).encode()) + return self + + def __exit__(self, *a): + try: + if self.fd is not None: + os.close(self.fd) + os.remove(self.path) + except OSError: + pass + + +def _alive(pid): + try: + if os.name == "nt": + r = run(["tasklist", "/FI", "PID eq %d" % pid], timeout=30) + return str(pid) in (r.stdout or "") + os.kill(pid, 0) + return True + except Exception: + return False + + +def collect(paths, recursive): + out = [] + for p in paths: + if os.path.isfile(p): + out.append(p) + elif os.path.isdir(p): + if recursive: + for dp, _, fn in os.walk(p): + out += [os.path.join(dp, f) for f in fn] + else: + out += [os.path.join(p, f) for f in os.listdir(p)] + else: + log("!! not found: %s" % p) + keep = [] + for f in sorted(out): + e = os.path.splitext(f)[1].lower() + if e in SKIP_EXT or e not in (VIDEO_EXT | AUDIO_EXT): + continue + if "_src" in os.path.basename(f) or ".part." in f: + continue + keep.append(f) + return keep + + +def main(): + ap = argparse.ArgumentParser(description=__doc__, + formatter_class=argparse.RawDescriptionHelpFormatter) + ap.add_argument("paths", nargs="+") + ap.add_argument("--out") + ap.add_argument("--recursive", "-r", action="store_true") + ap.add_argument("--codec", choices=["auto", "hevc", "x264"], default="auto") + ap.add_argument("--cq", type=int, default=27, help="NVENC HEVC quality (default 27)") + ap.add_argument("--crf", type=int, default=20, help="x264 CRF (default 20)") + ap.add_argument("--preset", default="medium") + ap.add_argument("--nvenc-preset", default="p6") + ap.add_argument("--audio", default="256k") + ap.add_argument("--channels", type=int, default=2, help="0 = keep the source layout") + ap.add_argument("--cpu", action="store_true", help="force libx265/libx264 (no NVENC)") + ap.add_argument("--overwrite", action="store_true") + ap.add_argument("--keep", action="store_true", help="keep the input file") + ap.add_argument("--dry-run", action="store_true") + ap.add_argument("--no-decode-check", action="store_true") + ap.add_argument("--lock", default=os.path.join(os.path.expanduser("~"), + ".transcode.lock")) + a = ap.parse_args() + + # --out is a DIRECTORY; create it up front so the free-space probe and the encode + # both have a real path to work with (os.path.dirname of a not-yet-made dir is None, + # and shutil.disk_usage raises WinError 3 on a path that does not exist). + if a.out: + os.makedirs(a.out, exist_ok=True) + if not a.dry_run: + os.makedirs(os.path.dirname(os.path.abspath(a.lock)) or ".", exist_ok=True) + + files = collect(a.paths, a.recursive) + if not files: + raise SystemExit("nothing to do: no video/audio files found in %s" % a.paths) + + use_nvenc = (not a.cpu) and has_nvenc() + log("=== transcode: %d file(s) nvenc=%s codec=%s dry=%s" + % (len(files), use_nvenc, a.codec, a.dry_run)) + + done = {"ok": 0, "skipped": 0, "failed": 0} + plans = [] + for src in files: + info = probe(src) + if not info: + log("SKIP %s: unreadable" % os.path.basename(src)); done["skipped"] += 1; continue + vmode, amode, reason = plan_for(info, src, a.codec) + if vmode is None: + log("SKIP %s: %s" % (os.path.basename(src), reason)); done["skipped"] += 1; continue + plans.append((src, info, vmode, amode, reason)) + + if a.dry_run: + for src, info, vmode, amode, reason in plans: + log("PLAN %s -> %s | video=%s audio=%s | %s" + % (os.path.basename(src), os.path.basename(out_path(src, a)), + vmode, amode, reason)) + log("=== dry-run: %d to convert, %d already fine" % (len(plans), done["skipped"])) + return + + with Lock(a.lock): + for i, (src, info, vmode, amode, reason) in enumerate(plans, 1): + dst = out_path(src, a) + probe_dir = os.path.dirname(os.path.abspath(dst)) or "." + os.makedirs(probe_dir, exist_ok=True) + free = shutil.disk_usage(probe_dir).free / 2 ** 30 + if free < MIN_FREE_GB: + log("!!! ABORT: %.1f GB free, need %d GB - %d item(s) not attempted" + % (free, MIN_FREE_GB, len(plans) - i + 1)) + break + log("--- %d/%d %s | video=%s audio=%s | %s" + % (i, len(plans), os.path.basename(src), vmode, amode, reason)) + if os.path.exists(dst) and not a.overwrite: + log("SKIP: %s exists (use --overwrite)" % os.path.basename(dst)) + done["skipped"] += 1; continue + + part = dst + ".part.mp4" + cmd = build_cmd(src, part, info, vmode, amode, a, use_nvenc) + t0 = time.time() + r = run(cmd) + if r.returncode != 0: + log("!! ffmpeg failed rc=%s: %s" % (r.returncode, (r.stderr or "").strip()[-300:])) + if os.path.exists(part): + os.remove(part) + done["failed"] += 1; continue + el = time.time() - t0 + + ok, msg = verify(info, part, vmode, amode, check_decode=not a.no_decode_check) + if not ok: + log("!! verify failed: %s" % msg) + os.remove(part) + done["failed"] += 1; continue + + os.replace(part, dst) + back = probe(dst) + if not back or abs(back["dur"] - info["dur"]) > max(1.0, info["dur"] * 0.01): + log("!! READ-BACK MISMATCH after writing %s - input left untouched" % dst) + done["failed"] += 1; continue + + if not a.keep and os.path.abspath(src) != os.path.abspath(dst): + try: + os.remove(src) + except OSError: + pass + sz = os.path.getsize(dst) + log(" ok %s %.0f MB (%.0f%% of source) in %.0fs" + % (os.path.basename(dst), sz / 1e6, 100.0 * sz / max(info["size"], 1), el)) + done["ok"] += 1 + + log("=== done: ok=%d skipped=%d failed=%d" % (done["ok"], done["skipped"], done["failed"])) + if done["failed"]: + sys.exit(1) + + +if __name__ == "__main__": + main()