The gate failed a perfectly good encode (min 0.2552, mean 0.9841) because it demanded a per-frame
floor of 0.985 for EVERY frame. Two measured facts say that is the wrong test:
* SSIM collapses on near-flat frames. Two almost-black frames of a fade score ~0.25 while their
neighbours score 0.998, so a minimum over all frames condemns good encodes. The gate now measures
each source frame's luma std and drops the flat ones from the verdict (reported, never hidden;
if more than half the window is flat the exclusion is skipped so the gate still means something).
* A defect has a SHAPE. Measured distributions on this library: good encodes 0 of 1500 frames below
0.95; the file that triggered this 5 of 1500 below 0.85 (0.33%, isolated) at a 0.984 mean; the
genuine decoder-defect files 44 of 1499 (2.9%) with the mean falling. So the verdict is now
'mean < 0.98 OR more than 1% of frames below 0.85' -> fail; a handful of scattered low frames at a
healthy mean is logged as fades, not a defect - and the log line carries the counts, so the
decision is auditable.
A read of the RIGHT LENGTH can still be wrong on this share - RaiDrive reports stale sizes, and a
full-file listing read here stopped halfway with no error at all (it looked exactly like a
truncated file). A corrupt staged source would be silently encoded into a corrupt output, so the
copy now compares a hash of what it wrote against a fresh read of the share copy (files over 64 MB,
one extra pass - cheap next to the encode it protects). Today's evidence: a 921 MB snapshot original
threw 'Error splitting the input into NAL units' in one pass and decoded perfectly in the next.
Two bugs the library run exposed on real files:
* An input seek is not a faithful decode. On an H.264 source with open GOPs the frames after the
landing point decode differently from a sequential decode - the SAME file compared with itself,
once seeked and once decoded in order, scored min 0.25 / mean 0.82, and the gate condemned good
encodes because of it. Both sides are now decoded sequentially with the window cut by the trim
filter, which is also the only decode guaranteed to match what the encoder itself saw.
* Staleness of the run lock. Killing a background session kills the wrapper but not the python
underneath, so a live run's lock survives and the next start is correctly refused (it looked like
a pid-reuse bug; it was not). The lock now carries a heartbeat, staleness is decided by age as
well as pid, and a takeover that cannot delete a file the OS still holds overwrites it in place.
Two lessons from chasing one stubborn file. A 60 s stream copy (-ss X -c copy) of an H.264 source
with B-frames lost 230 of its 1500 frames and shifted the content, so every encode of that cut
looked broken (SSIM minimum 0.25) - harnesses must sample with byte copies, never stream copies,
and must prove themselves against a file compared with itself. And when a single file resists
diagnosis, park it and report it instead of stalling a library run on one item.
Two failures from today's library run, both caught, both now handled:
1. The picture gate compared frames by INDEX after an input seek. A seek snaps to a keyframe, and
it snapped to a DIFFERENT keyframe for the source and for the encode - measured: two 1500-frame
decodes whose start times differed by ~10 s, so every frame met the wrong one and a perfectly
good encode scored min SSIM 0.25 (and a file with a hole in its source's timeline scored 0.45).
The window is now cut with the trim filter on absolute PTS (-copyts), so both sides cover
exactly the same interval no matter where the seek lands; a self-comparison then measures
lo=1.0/mean=1.0 (validated), and a frame-count difference >1% is reported as 'gate NOT
APPLICABLE' rather than failing a file whose source is simply missing frames.
2. RaiDrive/Dokan returned from a 403 MB write with NO error while the remote file held 12.5 MB,
and surfaced 'the operation didn't complete within specific time limit' afterwards. The
SHA-256 read-back caught it (nothing was replaced), and it now retries the upload instead of
failing outright. Uploads use 4 MB chunks with a periodic flush; source copies retry 5x with a
longer backoff and smaller chunks.
The old runner (fixed -cq 27 + fixed 256k audio + maxrate at 1.5x the FILE bitrate) left 54 files
BIGGER than their originals - the worst at 124.8%. Re-encoding those from their current HEVC would
be second-generation loss, so the source must be the pre-conversion original; a dated snapshot of
the library holds it (582 files / 324 GiB measured).
A file already HEVC is now re-done only when it is bigger than its original (>1.01x), which makes
the run idempotent with no extra state. The tier and every gate use the ORIGINAL, dst stays the
live path, and os.remove(src) is suppressed whenever the source came from the origin tree so the
archive of originals can never be deleted by a replace.
The previous runner encoded everything at a fixed -cq 27 with -maxrate = 1.5x the FILE bitrate,
which is how a library ended up at 111% of its source size. This replaces the encode step with the
policy the skill documents, and adds the gates that make it a guarantee:
* classify by bpp: <0.025 copy the video (bit-identical), 0.025-0.040 hevc_nvenc, >0.040 libx265
* -maxrate = 1.0x the source's VIDEO bitrate (never the file total) + a post-encode assertion
* SSIM gate: per-frame comparison of a 60 s window against the source, retry once at +2 quality
if it fails, then fail the item (the original is never touched)
* audio: copy when AAC, else AAC at the SOURCE's bitrate (a fixed 256k cancelled the video saving)
* x265 tier pinned to the E-cores; --tier to run tiers separately; --keep-local for a pilot that
touches nothing
* defaults cq31 / crf26 from the measurement below, not cq27
Measured on three real sources (60 s window, video-only): cq27 GROWS two of three files (115%,
119%); cq31 lands at 43-74%; x265 crf26 at 34-67% with min SSIM 0.988-0.993 - and x265 beats NVENC
by 10-22% at matched SSIM. Pinning min SSIM >= 0.996 leaves only cq27, which is bigger than the
source, so ~0.99 is the usable quality frontier and the owner gets to make that call explicitly
(references/hevc-conversion-policy.md section 11, SKILL.md scope section).
hoelee