diff --git a/public/banners/every-player-stuttered-the-file-was-fine.png b/public/banners/every-player-stuttered-the-file-was-fine.png new file mode 100644 index 0000000..e6ef6b3 Binary files /dev/null and b/public/banners/every-player-stuttered-the-file-was-fine.png differ diff --git a/public/og/every-player-stuttered-the-file-was-fine.png b/public/og/every-player-stuttered-the-file-was-fine.png new file mode 100644 index 0000000..0858afc Binary files /dev/null and b/public/og/every-player-stuttered-the-file-was-fine.png differ diff --git a/scripts/banner-gen/generate.mjs b/scripts/banner-gen/generate.mjs index d66d0c4..2014eba 100644 --- a/scripts/banner-gen/generate.mjs +++ b/scripts/banner-gen/generate.mjs @@ -1149,6 +1149,24 @@ BANNERS['putting-a-front-door-on-an-app-you-cant-modify'] = { ], }; +BANNERS['every-player-stuttered-the-file-was-fine'] = { + titlebar: "root@media - the file was blameless", + lines: [ + { t: 'prompt', text: "$" }, { t: 'cmd', text: "tsconv --file 5YkGHtaduGU" }, + { t: 'dim', text: "container: 5818/5818 frames @ 0.040000 s" }, { t: 'dim', text: "decode: rc=0, zero errors printed" }, + { t: 'err', text: "pictures: 106 frames wrong (SSIM 0.33)" }, { t: 'prompt', text: "$" }, + { t: 'cmd', text: "-hwaccel cuda -c:v av1" }, { t: 'ok', text: "0 wrong frames SSIM 1.000000" }, + { t: 'hl', text: "gate: structure AND content, then replace" }, { t: 'dim', text: "65/65 affected files re-encoded" }, + ], + flow: [ + { n: '1', label: "decode" }, + { n: '2', label: "encode" }, + { n: '3', label: "verify" }, + { n: '4', label: "content" }, + { n: '5', label: "replace" }, + ], +}; + // ---------- read frontmatter ---------- const postPath = join(ROOT, 'src', 'content', 'posts', `${slug}.md`); let category = 'devops'; diff --git a/scripts/og-gen/generate.mjs b/scripts/og-gen/generate.mjs index b8c382a..6bb770a 100644 --- a/scripts/og-gen/generate.mjs +++ b/scripts/og-gen/generate.mjs @@ -349,6 +349,12 @@ TERMINALS['putting-a-front-door-on-an-app-you-cant-modify'] = `
$docker compose up -d (gateway takes the public port)→ front page 200 ✓
$curl -s /index.html | grep -o lang=→ zh-CN ✓
`; +TERMINALS['every-player-stuttered-the-file-was-fine'] = ` +
$ffmpeg -v error -i out.mp4 -f null - -> 0 errors
+
 106/5819 frames show the WRONG picture
+
$-c:v av1_cuvid -> -hwaccel cuda -c:v av1
+
 0 wrong frames SSIM 1.000000-> fixed
`; + // ---------- read frontmatter ---------- const postPath = join(ROOT, 'src', 'content', 'posts', `${slug}.md`); if (!existsSync(postPath)) { diff --git a/src/content/posts/every-player-stuttered-the-file-was-fine.md b/src/content/posts/every-player-stuttered-the-file-was-fine.md new file mode 100644 index 0000000..759b021 --- /dev/null +++ b/src/content/posts/every-player-stuttered-the-file-was-fine.md @@ -0,0 +1,189 @@ +--- +title: "Every Player Stuttered on the Same Video. The File Was Blameless." +description: "Same HEVC file, same stutter in VLC, Chrome and an iPad — offline. The container timing was perfect. The pictures were not: NVDEC's legacy AV1 wrapper was returning frames from the wrong timestamps." +pubDate: 2026-10-08 +category: devops +tags: [ffmpeg, nvdec, hevc, transcoding, video] +ogImage: /og/every-player-stuttered-the-file-was-fine.png +banner: /banners/every-player-stuttered-the-file-was-fine.png +draft: false +--- + +## The question I could not answer + +A converted video stuttered. Not badly — a hitch every few seconds, then it +carried on. The kind of thing you blame on Wi-Fi. + +Except it happened on an iPad Pro M1 playing the file **offline**, in VLC, +and in Chrome on a desktop. Different decoders, different platforms, same +periodic hitch. The original download played perfectly everywhere. So the +network was out, the device was out, and the file was in. + +I spent a long time measuring the file, and every measurement said the file +was fine. It was not fine. It was a faithful re-encode of a decoder bug. + +## Step 1: measure the container, not the picture + +The converted file is HEVC Main 8-bit in MP4 (`hvc1`), 3840×2160, 25 fps, +yuv420p, bt709, average 15.49 Mbps, 232.8 seconds, 5819 frames, `moov` +before `mdat`. + +Everything about its timeline is textbook: + +```text +frame intervals 5818 of 5818 = exactly 0.040000 s +duplicate PTS 0 +negative PTS 0 +gaps > 1.5x median 0 +edit list (ctts) 5796 entries (not a flat list) +has_b_frames 2 +A/V start offset 0.000 s +B-frame pattern I B B B P B B B ... (744 groups, all 3 B-frames) +``` + +A flat `ctts` — every presentation offset set to zero — is the classic way a +re-encode ends up with PTS sitting in decode order, and it makes players +drop roughly a quarter of the frames. That is what I expected to find. It +was not there. + +Full decode, both verbosity levels: + +```bash +ffmpeg -v warning -i converted.mp4 -f null - +ffmpeg -v error -i converted.mp4 -f null - +``` + +Both exit 0, both print nothing, 174 seconds of wall time for 232 seconds of +video. No corrupt frames, no missing references, no non-monotonic DTS. + +The GOP looked different from the source (24 keyframes, a constant 10.0 s, +because `hevc_nvenc` defaults to `-g 250`; the source has 59 keyframes at an +average 3.88 s) and the bitrate was 17% higher than the source — but the +source's own peak was higher than the output's, so neither explains a hitch +every few seconds. + +**Everything measurable in the container was correct.** Which is the point +where you have to stop measuring the container. + +## Step 2: compare pictures, from two different decoders + +If the timing is right and the file still looks wrong, the question becomes +whether the *pictures* are right. I compared the source file decoded two +ways, frame by frame, with SSIM: + +```bash +# software decode of the source +ffmpeg -i source.mkv -vf "scale=640:360" -f rawvideo -pix_fmt yuv420p sw.yuv + +# the same frames through NVDEC's AV1 path +ffmpeg -c:v av1_cuvid -i source.mkv -vf "scale=640:360" -f rawvideo -pix_fmt yuv420p hw.yuv +``` + +106 frames out of 5819 disagree — 1.8% — with SSIM as low as **0.330**. Not +uniform noise: the interval histogram peaks at about every 30 frames. + +Two control tests made it worse and better at the same time: + +- **Deterministic.** Decoding the same source twice through `av1_cuvid` +gives SSIM 1.000000 between the two runs. This is not a race; every +conversion will weld the same bad frames into the output. +- **Not all AV1.** A 1080p AV1 sample decoded identically through both paths +(SSIM 1.000000). A second 4K AV1 stream did not. It is stream-dependent. + +Then the decisive comparison — which side does the converted file match? + +| Comparison | Mean SSIM | Minimum | Frames below 0.98 | +|---|---|---|---| +| Converted HEVC vs software decode of the source | 0.99091 | **0.330** | **106** | +| Converted HEVC vs NVDEC decode of the source | **0.99742** | 0.991 | **0** | + +The converted file agrees with the *wrong* decode, frame for frame. It is +not a bad encode. It is an accurate encode of the wrong pictures. + +A single extracted frame shows it plainly: frame 546 through software decode +is a wide shot of a choir; through `av1_cuvid` it is a close-up from a +different moment in the video — and the converted file has the close-up. + +## Step 3: the fix is one line + +The bug is not the hardware. It is the legacy decoder wrapper. `av1_cuvid` +is an old-style wrapped decoder; the modern hardware path goes through +`hwaccel`: + +```bash +# before +-c:v av1_cuvid + +# after +-hwaccel cuda -hwaccel_output_format cuda -c:v av1 +``` + +Same stream, same source, 1500 frames compared against the software +reference: + +```text +-hwaccel cuda -c:v av1 SSIM 1.000000 (bit-identical, every frame) +vp9_cuvid vs libvpx-vp9 SSIM 0.99991 (0 frames below 0.98 - unaffected) +``` + +Encoding speed did not change: a 30-second 4K slice in 24.1 s, 1.24× +realtime either way. + +To be precise about the scope: this is a bug in the `*_cuvid` wrapper for +**AV1** on this driver/library combination, reproducible on demand. It says +nothing about your GPU's AV1 decoder — the hardware path is clean. + +## Step 4: make it impossible to ship again + +The uncomfortable part is that my pipeline *had* a verification step. It +checked the duration, the resolution, the codec, the frame rate and the +frame rhythm — all container properties. Every one of them passed on a file +with 106 wrong frames. + +So the pipeline is now five steps, and the fourth one is the picture check: + +```text +decode -> encode -> verify structure -> verify content -> replace the file +``` + +The content gate compares the output's frames against a software decode of +the source (both sides decoded sequentially, cropped by timestamp), and uses +the defect's *shape* rather than a single-frame threshold: + +- fail if any frame falls below 0.85 +- fail if more than 1% of frames fall below 0.95 +- fail if the mean falls below 0.98 +- ignore near-flat frames (SSIM is meaningless on frames that are almost +entirely black — two nearly-black frames can score 0.25 while being visually +identical) + +Those numbers are calibrated against measured data, not intuition. Good +encodes in this library sit at 0 frames of 1500 below 0.95. The defective +files sit at 2.9% of frames below 0.85 with a depressed mean. A perfectly +good 4K60 encode of a dance video once measured min 0.949 — a single frame +below 0.95, at a hard cut — which a single-frame threshold would have +rejected forever. + +Scoped to the library: 75 files had AV1 originals in the snapshots, 65 of +them (87%) failed the content gate. All 65 were re-encoded from the pristine +snapshot originals — never from the already-corrupted file — and every one +landed with min SSIM 0.997–0.999. + +## What I would do differently + +1. **"It decodes without errors" is not a quality check.** Decoding a frame +successfully says nothing about whether it is the *right* frame. The picture +comparison is the check that found it. 2. **Compare against a second +decoder, not against your expectations.** A software decode is free and it +is the reference. If your hardware path disagrees with it, you have found +something worth understanding. 3. **Verify the output against the source, +not the source's metadata.** 4. **Do not assume the modern and legacy +hardware paths are the same code.** `av1_cuvid` and `-hwaccel cuda -c:v av1` +reach the same silicon through different plumbing. A bug in one is not a +verdict on the hardware. 5. **A gate that fails safe is worth more than a +gate that never fails.** My first gate rejected that 4K60 encode. The file +was fine and the gate was wrong — but the failure cost one wasted encode, +not a bad file in the library. + +The stutter is gone. Not because the encoder got better, but because the +decoder stopped lying. diff --git a/src/content/posts/zh/every-player-stuttered-the-file-was-fine.md b/src/content/posts/zh/every-player-stuttered-the-file-was-fine.md new file mode 100644 index 0000000..e8ce0b1 --- /dev/null +++ b/src/content/posts/zh/every-player-stuttered-the-file-was-fine.md @@ -0,0 +1,134 @@ +--- +title: "每个播放器都在同一个视频上卡顿，而文件本身是无辜的" +description: "同一个 HEVC 文件，在 VLC、Chrome 和 iPad 上（离线）都出现同样的周期卡顿。容器层时间轴完美无缺，画面却不是——NVDEC 的旧版 AV1 包装器返回了错误时间点的帧。" +pubDate: 2026-10-08 +category: devops +tags: [ffmpeg, nvdec, hevc, transcoding, video] +ogImage: /og/every-player-stuttered-the-file-was-fine.png +banner: /banners/every-player-stuttered-the-file-was-fine.png +draft: false +--- + +## 一个我回答不了的问题 + +有个转好的视频会卡顿。不严重——每隔几秒顿一下，然后继续放。这种毛病你通常会怪到 Wi-Fi 头上。 + +问题是它出现在一台 **离线**播放的 iPad Pro M1 上、出现在 VLC 里、也出现在桌面版 Chrome 里。不同的解码器、不同的平台、同样的周期性卡顿。而原始下载的视频在所有地方都播放正常。所以网络被排除、设备被排除，剩下的嫌疑只有那个文件。 + +我花了很长时间测量这个文件，所有测量都说它是好的。它并不是好的。它是**对一个解码器 bug 的忠实重编码**。 + +## 第一步：先量容器，而不是量画面 + +这个转好的文件是 MP4 容器里的 HEVC Main 8-bit（`hvc1`）、3840×2160、25 fps、yuv420p、bt709、平均 15.49 Mbps、232.8 秒、5819 帧，`moov` 在 `mdat` 之前。 + +它的时间轴是教科书级别的干净： + +```text +帧间隔 5818 个全部 = 恰好 0.040000 秒 +重复 PTS 0 +负值 PTS 0 +超过中位数 1.5 倍的缺口 0 +edit list (ctts) 5796 条（不是扁平表） +has_b_frames 2 +音视频起始偏移 0.000 秒 +B 帧图案 I B B B P B B B ……（744 组，每组都是 3 个 B） +``` + +扁平的 `ctts`——把每个呈现偏移都设成零——是重编码后 PTS 停留在解码顺序上的经典症状，会让播放器丢掉大约四分之一的帧。这正是我预期会找到的东西。它不在那里。 + +完整解码，两个详细级别都跑： + +```bash +ffmpeg -v warning -i converted.mp4 -f null - +ffmpeg -v error -i converted.mp4 -f null - +``` + +两条都以 0 退出、什么都不打印，232 秒的视频用了 174 秒跑完。没有损坏帧、没有缺失参考帧、没有非单调 DTS。 + +GOP 确实和源不同（24 个关键帧、恒定 10.0 秒，因为 `hevc_nvenc` 默认 `-g 250`；源是 59 个关键帧、平均 3.88 秒），码率也比源高 17%——但源自身的峰值比输出的峰值还高，所以两者都解释不了"每隔几秒顿一下"。 + +**容器里所有可测的东西都是正确的。** 而这恰恰是"你就该停止测量容器"的那个点。 + +## 第二步：用两个不同的解码器去比画面 + +如果时间轴是对的、文件看起来却仍是错的，问题就变成：**画面**本身对不对。我把源文件用两种方式解码，再逐帧做 SSIM 对比： + +```bash +# 软件解码源文件 +ffmpeg -i source.mkv -vf "scale=640:360" -f rawvideo -pix_fmt yuv420p sw.yuv + +# 同一批帧走 NVDEC 的 AV1 路径 +ffmpeg -c:v av1_cuvid -i source.mkv -vf "scale=640:360" -f rawvideo -pix_fmt yuv420p hw.yuv +``` + +5819 帧里有 **106 帧**不一致——占 1.8%——SSIM 最低到 **0.330**。不是均匀的噪声：间隔直方图的峰值大约落在每 30 帧一次。 + +两个对照实验，一个让情况更糟、一个让情况更清楚： + +- **确定性。** 同一个源用 `av1_cuvid` 解两次，两次之间 SSIM 是 1.000000。这不是竞争条件；每一次转换都会把同一批坏帧焊进输出里。 +- **不是所有 AV1 都中招。** 一个 1080p 的 AV1 样本两条路径解出来完全一致（SSIM 1.000000），而另一个 4K AV1 流不一致。这跟具体的流有关。 + +然后是决定性的对比——转好的文件到底跟哪一边一致？ + +| 对比 | 平均 SSIM | 最低 | 低于 0.98 的帧数 | +|---|---|---|---| +| 转好的 HEVC vs 源的软件解码 | 0.99091 | **0.330** | **106** | +| 转好的 HEVC vs 源的 NVDEC 解码 | **0.99742** | 0.991 | **0** | + +转好的文件和**错误的那一份**解码逐帧吻合。它不是一次糟糕的编码，它是**对错误画面的一次准确编码**。 + +抽出一帧就能看得很清楚：第 546 帧在软件解码里是合唱团的远景；在 `av1_cuvid` 里是另一个时刻的特写镜头——而转好的文件里是那张特写。 + +## 第三步：修复只要一行 + +Bug 不在硬件，在旧的解码器包装器。`av1_cuvid` 属于老式的 wrapper 解码路径；现代的硬件路径要走 `hwaccel`： + +```bash +# 修复前 +-c:v av1_cuvid + +# 修复后 +-hwaccel cuda -hwaccel_output_format cuda -c:v av1 +``` + +同一个流、同一个源，1500 帧对软件参考逐帧比较： + +```text +-hwaccel cuda -c:v av1 SSIM 1.000000 （逐帧逐位一致） +vp9_cuvid vs libvpx-vp9 SSIM 0.99991 （0 帧低于 0.98——不受影响） +``` + +编码速度没有变化：一段 30 秒的 4K 素材 24.1 秒编完，两种路径都是 1.24× 实时。 + +范围要说准确：这是这套驱动/库组合下 **AV1** 的 `*_cuvid` 包装器的 bug，可稳定复现。它与你显卡的 AV1 解码器无关——现代硬件路径是干净的。 + +## 第四步：让它再也不可能悄悄发出去 + +最让人不安的地方在于：我的流水线**本来就有**验证步骤。它检查时长、分辨率、编码器、帧率和帧节奏——全是容器属性。这些检查在一个带着 106 帧错误画面的文件上**全部通过**。 + +所以现在流水线是五步，第四步才是画面检查： + +```text +解码 -> 编码 -> 结构验证 -> 内容验证 -> 替换文件 +``` + +内容闸门把输出帧与源的软件解码逐帧对比（两侧都顺序解码、按时间戳裁剪取窗口），并且判的是缺陷的**形状**，而不是某一帧的硬线： + +- 任何一帧低于 0.85 → 失败 +- 低于 0.95 的帧超过 1% → 失败 +- 均值低于 0.98 → 失败 +- 剔除近平坦帧（对几乎全黑的帧，SSIM 没有意义——两帧几乎全黑也能算出 0.25，而它们在视觉上完全相同） + +这些阈值是拿实测数据标定出来的，不是凭直觉。这个素材库里好的编码在 1500 帧里有 0 帧低于 0.95；有缺陷的文件有 2.9% 的帧低于 0.85 且均值下滑。而一段完全正常的 4K60 舞蹈视频曾经测出最低 0.949——一个硬切处只有 1 帧低于 0.95——用单帧硬线判的话，它会被永久拒绝。 + +范围：快照里有 AV1 原档的共 75 个文件，其中 **65 个（87%）**没通过内容闸门。这 65 个全部用**快照里的干净原档**重转（绝不用已经被弄坏的文件），每一个都以 min SSIM 0.997–0.999 落地。 + +## 我会怎么做得不一样 + +1. **"解码没有报错"不是质量检查。** 一帧能解码成功，跟它是不是**正确的那一帧**毫无关系。找出问题的是画面比对。 +2. **跟第二个解码器比，而不是跟你的预期比。** 软件解码是免费的，而且它就是参考基准。如果你的硬件路径跟它不一致，你就找到了值得搞清楚的东西。 +3. **验证输出要对着源，而不是对着源的元数据。** +4. **别以为现代硬件路径和旧包装器是同一条代码路径。** `av1_cuvid` 和 `-hwaccel cuda -c:v av1` 走的是不同的管道、到的是同一块硅片。其中一个有 bug，不等于对硬件下判决。 +5. **一个"失败得安全"的闸门，比一个从不失败的闸门值钱得多。** 我的第一版闸门拒绝了那段 4K60 编码，文件是好的、闸门是错的——但这个错误的代价只是白烧一次编码，而不是让一个坏文件进了库。 + +卡顿消失了。不是因为编码器变好了，而是因为解码器不再说谎了。