The GPU tier sat frozen for 1.5 hours: the user updated the NVIDIA driver while a hevc_nvenc encode
was running, which wedges in-flight GPU work with no error at all - the process stays alive, the
output stops growing, the encoder reads 0%, and nothing anywhere reports a problem. A multi-day
unattended run cannot park like that, so every long command that writes a file (the encode, and the
gate's decodes) is now watched: no growth for 300 s and the wedged process is killed and reported as
a failure for the normal retry path to pick up. Self-checked with a process that writes once and
then sleeps: killed at ~35 s for a 20 s stall window.
hoelee
5c911f5450