library_run: --force-tier (explicit override of the bpp rule) for the control plane
An owner decision can say a LEAN source is still worth re-encoding (measured: x265 crf27 lands at 33-53% of the source video bitrate at SSIM mean >=0.996). --tier cannot express that: it SKIPS any file whose classification disagrees. --force-tier applies the requested tier instead, with two guards that stay: copy is refused for a VP9/AV1 source (iOS cannot play the result) and a re-encode is refused for a file that is already HEVC hvc1 (that is the --overwrite decision).
This commit is contained in:
@@ -75,7 +75,7 @@ structural gate and confirm on the Apple device.
|
||||
|
||||
## 6. Related work
|
||||
|
||||
- **`hoelee-tubesync-management`** — the TubeSync archive this tool was extracted from. It
|
||||
- **`hoelee-resource-transcode`** — the TubeSync archive this tool was extracted from. It
|
||||
owns the DB-driven pipeline (`tsconv.py` / `tsfix.py`) and the deeper frame-timing
|
||||
forensics. `tsconv.flat_timing()` is that repo's copy of the detector: **keep the two in
|
||||
step**, and note that the missing-`ctts` shape was once missed by three duplicated copies
|
||||
|
||||
@@ -115,13 +115,13 @@ live = HEVC hvc1 **且**音轨是 AAC（真能播）**且**本次是从快照原
|
||||
4. **安装只用 ffprobe 元数据复核** → RaiDrive 返回**缓存属性**，rename 没落地也说成功（实测"ok 1085→478 MB"而线上仍是原档）→ 改**安装后回读哈希**。
|
||||
5. **源拷贝只校验字节数** → 加 **SHA-256 比对共享盘副本**（长度对内容错是真风险）。
|
||||
6. **probe 单次失败即判文件失败** → 加**重试 3 次**。
|
||||
7. **看门狗用 stderr 管道**（本轮最贵的 bug）→ ffmpeg `-stats` 每 0.5 s 一行，64 KB 管道几分钟塞满 → ffmpeg 阻塞在 write() → 输出不增长 → 看门狗**误杀好编码**（整夜 ~270 杀、只转成 24 个、GPU 0%）。修：**stderr 写文件**，阈值 5→15 分钟。自检脚本 `D:\dev\hoelee-tubesync-management\cache\watchdog_selfcheck.py`。
|
||||
7. **看门狗用 stderr 管道**（本轮最贵的 bug）→ ffmpeg `-stats` 每 0.5 s 一行，64 KB 管道几分钟塞满 → ffmpeg 阻塞在 write() → 输出不增长 → 看门狗**误杀好编码**（整夜 ~270 杀、只转成 24 个、GPU 0%）。修：**stderr 写文件**，阈值 5→15 分钟。自检脚本 `D:\dev\hoelee-resource-transcode\cache\watchdog_selfcheck.py`。
|
||||
8. 看门狗**保留**（它要防的是真实场景：用户更新 NVIDIA 驱动会把在跑的 NVENC 冻死 1.5 小时且不报错）。
|
||||
9. **Hermes 后台进程会随会话一起被杀** —— 两个档在 10:37 同时死在编码中，日志里没有 `PASS-DONE`；改脱管启动（见上）。
|
||||
|
||||
## 未完成 / 待办
|
||||
|
||||
- [ ] 两档跑完 → **全库落地审计**：`D:\dev\hoelee-tubesync-management\cache\install_audit.py`（核对线上文件真实大小 + 编码器；曾一次抓到"报告成功没落地"）+ `copy_land_check.py`（copy 档用音轨 aac/mp3 判定）。
|
||||
- [ ] 两档跑完 → **全库落地审计**：`D:\dev\hoelee-resource-transcode\cache\install_audit.py`（核对线上文件真实大小 + 编码器；曾一次抓到"报告成功没落地"）+ `copy_land_check.py`（copy 档用音轨 aac/mp3 判定）。
|
||||
- [ ] 失败项：无需清单，新一轮自动重做（失败时文件原样）。
|
||||
- [ ] `2022年/0723 《走出强迫症》/走出强迫症.mp4` 用新下载（`D:\Download\0723走出强迫症.mp4`，618,696,954 B）已转好装好（450,502,893 B，HEVC+AAC，SSIM min 0.997，时长 3399.81 s）✓。**快照里那份仍是坏的**（618,697,011 B，moov 缺失）→ 待定：是否用干净副本替换快照里的坏档，以免将来"从原档重转"踩坑。
|
||||
- [ ] 目标文件夹里曾出现一个 0 字节的 `0723走出强迫症.mp4`（现已不在）—— 若是用户自建请告知命名约定。
|
||||
|
||||
+1
-1
@@ -121,5 +121,5 @@ Why this is safe to schedule:
|
||||
Before pointing it at a live library, `--dry-run` prints the plan and encodes nothing.
|
||||
|
||||
> **TubeSync specifically:** use that project's own DB-driven pipeline
|
||||
> (`hoelee-tubesync-management`: `tsconv.py` / `tsfix.py`) — it also updates the database
|
||||
> (`hoelee-resource-transcode`: `tsconv.py` / `tsfix.py`) — it also updates the database
|
||||
> rows. This tool is for everything outside it.
|
||||
|
||||
+19
-1
@@ -679,7 +679,18 @@ def process(src, args, dest=None, origin=None):
|
||||
vbps = src_video_bps(src, info)
|
||||
bpp = bpp_of(info, vbps)
|
||||
tier = tier_for_bpp(bpp)
|
||||
if args.tier and args.tier != tier:
|
||||
forced = getattr(args, "force_tier", None)
|
||||
if forced:
|
||||
# An explicit override, but never a silent one, and never one that breaks playability:
|
||||
# * copy on a VP9/AV1 source would ship an MP4 whose video iOS cannot decode;
|
||||
# * a re-encode of a file that already plays (HEVC hvc1) is the `--overwrite` decision.
|
||||
if forced == "copy" and info["vcodec"] not in ("h264", "avc1", "hevc"):
|
||||
return "SKIP force-tier copy refused: %s source cannot play on iOS" % info["vcodec"], None
|
||||
if forced != "copy" and info["vcodec"] == "hevc" and info["tag"] == "hvc1" and not T.unusable_timeline(info, src)[0]:
|
||||
return "SKIP force-tier re-encode refused: already HEVC hvc1 (use --overwrite)", None
|
||||
log(" force-tier %s (bpp rule said %s)" % (forced, tier))
|
||||
tier = forced
|
||||
elif args.tier and args.tier != tier:
|
||||
return "SKIP classified %s (run wants %s)" % (tier, args.tier), None
|
||||
expect_vc = "hevc" if tier != "copy" else (info["vcodec"] or "h264")
|
||||
log(" tier=%s bpp=%.4f src_video=%.0f kbps %sx%s@%.0f %s" % (
|
||||
@@ -878,6 +889,13 @@ def main():
|
||||
ap.add_argument("--pools", type=int, default=X265_POOLS, help="x265 threads (match the E-core mask)")
|
||||
ap.add_argument("--tier", choices=["copy", "nvenc", "x265"],
|
||||
help="only touch files classified into this tier (run tiers separately)")
|
||||
ap.add_argument("--force-tier", choices=["copy", "nvenc", "x265"], default=None,
|
||||
help="override the bpp classification for this run instead of skipping the "
|
||||
"file (the control plane needs it: an owner decision can say a LEAN "
|
||||
"source is still worth re-encoding, and `--tier x265` would otherwise "
|
||||
"refuse every lean file it disagrees about). Two guards stay in place: "
|
||||
"copy is refused for a source iOS cannot play (VP9/AV1), and a re-encode "
|
||||
"is refused for a file that is already HEVC hvc1.")
|
||||
ap.add_argument("--origin-root", default=None,
|
||||
help="tree holding the PRE-CONVERSION originals (e.g. a snapshot); a file that "
|
||||
"is already HEVC and BIGGER than its original is re-done from that "
|
||||
|
||||
Reference in New Issue
Block a user