Files

T

Hermes Agent 0cdb54f910 universal-video-transcode: any file in, one iPad+Android-playable MP4 out

Give it a path; it reads the source, picks the recipe, encodes, and verifies the
result before accepting it.

The problem it exists to remove: a file can drop ~23% of its frames in a browser
while r_frame_rate == avg_frame_rate reads EXACTLY equal, so every fps-based check
calls it healthy and it never enters a repair queue. The cause is deterministic and
header-readable -- the MP4 composition-offset table is unusable, either a single
constant ctts entry (PTS == DTS + const) or no ctts box at all (PTS == DTS) -- so the
frame timestamps are in decode order and the decoder's reordered output frames carry
the wrong times.

- three recipes chosen from the source FILE, never the filename: lossless -c copy when
  the timeline allows it, AAC-only fix when just the audio is wrong, full HEVC (or
  x264 for reach) when the timeline cannot be copied
- quality policy: never upscale; HEVC -cq 27 measured at SSIM 0.9937 ~= source size;
  audio copied when already AAC, else AAC 256k
- the two flags that must never be dropped: -fps_mode cfr and -pix_fmt yuv420p
- verified before acceptance: duration, resolution, codec, hvc1 tag, 8-bit, uniform
  cadence, no flat composition offsets, full decode pass, and a read-back after the write
- built for cron: lock file (two concurrent NVENC jobs starve one encoder and DROP
  FRAMES), idempotent, leaves the input untouched on failure, aborts below 5 GB free
- scripts/selftest.py builds both defect shapes and asserts the detector, including a
  Matroska case that must NOT be flagged

Measured on the file that prompted this: 23.0% of frames dropped before, 0.0% after.

2026-10-04 13:01:14 +08:00

4.3 KiB

Raw Blame History

AGENTS.md — entry point for any AI session touching this repo

This repo is the canonical source of the universal-video-transcode Hermes skill. Read this file first, then README.md, then the specific reference you need.

Owner: Lee Teong Hoe (Mr Hoelee) · [email protected] · WhatsApp +60 12-797 2969

1. What this is

One tool that turns any video/audio file into an MP4 that plays on both iPad and Android, at the best quality the target codec allows.

layer	what	where
the tool	`scripts/transcode.py` — path in, verified MP4 out	this repo
the gate	`scripts/selftest.py` — builds both defect shapes and asserts the detector	this repo
the skill	`SKILL.md` + `references/` — loads into Hermes	copied to the skill dir (§4)

There are no credentials in this project, so there is no SECRETS.md.

2. The non-negotiables (read before you change anything)

-fps_mode cfr on every video re-encode. Without it the encoder silently drops 0.5–1.3% of frames, leaving doubled frame intervals that play as judder. verify() fails an item whose output comes back non-uniform — never remove that check.
-tag:v hvc1 on every HEVC output, and -pix_fmt yuv420p. Apple refuses hev1 (ffmpeg's default) and non-8-bit formats lose device compatibility.
Never copy a video whose timeline is defective. -c copy cannot retime — ffmpeg will not rewrite stts/ctts under stream copy — so a flat input survives into the output. plan_for() decides this; do not "simplify" it into an unconditional copy.
One encoder at a time. The lock file is not optional hygiene: two concurrent NVENC jobs share one encoder unit, starve each other, and drop frames.
Run python scripts/selftest.py after touching the detector. It is the only thing standing between this tool and shipping files that stutter.
Never write encode output straight to a network share. Encode to local disk, verify, then copy the complete file across; a streaming .part write leaves a corrupt stub on any blip. If the destination is remote, keep the output local.

3. Layout

SKILL.md                                   the Hermes skill (installed per §4)
README.md                                  human-facing overview
scripts/transcode.py                       the tool: plan -> encode -> verify -> keep input on failure
scripts/selftest.py                        proves the detector; both defect shapes, no false positives
references/compatibility-matrix.md         per-platform codec/container facts with citations
references/verification-and-automation.md  the defect classes, the structural gate, the 2x2 proof

4. Installing the skill

cp -r . "$LOCALAPPDATA/hermes/skills/media/universal-video-transcode"

Keep this repo the source of truth and copy to the skill dir, not the reverse — edits made only in the skill dir are lost the next time it is reinstalled.

5. What "verified" means here

An output is accepted only if all of these hold, each read back from the file:

duration within 1% · resolution identical · expected video codec · hvc1 tag on HEVC · audio is AAC · 8-bit yuv420p · uniform cadence · no flat composition offsets · full decode exits 0 · read-back after the write matches.

Do not claim a fix without running the arbiter. measure_playback.py in the tubesync-selfhosted skill plays a file in real Chrome and reports getVideoPlaybackQuality().droppedVideoFrames — but note Chrome cannot decode HEVC, so an HEVC file reports 0 decoded / 0 dropped and that is not a pass. Judge HEVC by the structural gate and confirm on the Apple device.

hoelee-tubesync-management — the TubeSync archive this tool was extracted from. It owns the DB-driven pipeline (tsconv.py / tsfix.py) and the deeper frame-timing forensics. tsconv.flat_timing() is that repo's copy of the detector: keep the two in step, and note that the missing-ctts shape was once missed by three duplicated copies of exactly this logic. One implementation per repo.
references/verification-and-automation.md carries the measured 2x2 that proved the defect is container timing and not the bitstream. If you are about to argue that a bitstream property is the cause, read that first.

4.3 KiB Raw Blame History Unescape Escape