review: apply external consultant editorial fixes and update audit tracker to 74/91 approved
Deploy / build (push) Successful in 28s

This commit is contained in:
hoelee committed 2026-10-27 05:29:08 +08:00
1 parent 4e64bfc9ae
commit a7e683100b
56 files changed
+14496 -23169

No files matched your search

@@ -8,6 +8,7 @@ tags: ["docker", "cdp", "playwright", "youtube", "portainer", "automation"]
ogImage: "/og/uploading-large-videos-to-youtube-via-cdp.jpg"
banner: "/banners/uploading-large-videos-to-youtube-via-cdp.jpg"
draft: false
dynamicCta: true
---
Automating web uploads via headless browser or Chrome DevTools Protocol (CDP) works seamlessly for small JSON payloads, forms, and occasional screenshots. But when you attempt to upload a 100MB+ 1080p video file to YouTube through a remote browser session, standard browser automation approaches quickly collapse.
@@ -81,12 +82,15 @@ The real question becomes: **How do you get a 130MB file into the container file
Every Docker daemon exposes a container archive endpoint:
`PUT /containers/{id}/archive?path=/path/in/container`
If you manage your Docker host with Portainer (standard in Synology and unRaid homelabs), Portainer proxies this Docker socket endpoint behind an API key:
If you manage your Docker host with Portainer (a common control plane on self-hosted Synology and Unraid nodes), Portainer proxies this Docker socket endpoint behind an API key:
`PUT http://<host>:9000/api/endpoints/2/docker/containers/{container_id}/archive?path=/config/Downloads`
Here is a tested, production-ready staging script with proper error handling, parameter validation, and secure file modes:
<details>
<summary><code>stage_file_to_container.py</code> — Hardened Portainer Docker Archive staging script (50 lines)</summary>
```python
import io
import os
@@ -140,6 +144,8 @@ def stage_file_to_container(local_path: str, dest_filename: str) -> None:
raise RuntimeError(f"Portainer Archive API error {e.code}: {body}") from e
```
</details>
### Operational Caveats & Security Checklist
1. **Client RAM Footprint**: While Chrome experiences zero memory pressure, `io.BytesIO()` buffers the 130MB tar archive in the Python process memory. For multi-gigabyte video dumps, write the `.tar` to a temporary file and stream it using `requests` or `urllib3` chunked uploads.
2. **Pre-existing Destination Directory**: The Docker Archive API extracts *into* the directory specified by `?path=`. If `/config/Downloads` does not exist in the container image, the API returns HTTP 404/500 rather than creating the parent folders.
@@ -173,3 +179,11 @@ Because the file was already on disk inside the container, YouTube Studio reads
- **Never stream large binaries over WebSocket CDP**. Remote browsers are not designed to buffer multi-hundred-megabyte payloads in memory.
- **Stage files out-of-band**: Use Docker Archive PUT to drop files onto the container filesystem in seconds.
- **Inject via filesystem path**: Call `DOM.setFileInputFiles` with the container path for instantaneous, zero-overhead file attachments.
---
## Need rock-solid browser or media-publishing automation without API quota walls?
I architect and maintain self-hosted browser automation and media pipelines — from persistent Dockerised Chrome fleets driven over CDP to end-to-end video rendering and publishing workflows. If your team has a high-volume upload, portal workflow, or multi-step operation that breaks standard headless scrapers, I can build a deterministic pipeline on your own infrastructure.
Reach me at [[email protected]](mailto:[email protected]?subject=Browser%20and%20media%20pipeline%20automation) or WhatsApp [+60 12-797 2969](https://wa.me/60127972969), or see what I do at [hoelee.com](https://hoelee.com).