From 5d7382bb0229b33b7dc9349c8a994545be28f213 Mon Sep 17 00:00:00 2001 From: tcely Date: Wed, 11 Jun 2025 04:03:20 -0400 Subject: [PATCH 1/4] Create youtube-pot.md --- docs/youtube-pot.md | 45 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+) create mode 100644 docs/youtube-pot.md diff --git a/docs/youtube-pot.md b/docs/youtube-pot.md new file mode 100644 index 00000000..ee6b17a5 --- /dev/null +++ b/docs/youtube-pot.md @@ -0,0 +1,45 @@ +After #1005 was merged, you are welcome to try the new support for the plugin. + +The implementation mostly went as I planned, except for needing to use an IP address. + +> [!TIP] +> You can skip all of this `nginx` mess entirely by configuring the base URL the plugin tries to access in the settings file. +> +> ```python +> YOUTUBE_DEFAULTS = { +> 'extractor_args': { +> # old plugin versions looked under 'youtube' +> 'youtube': { +> 'getpot_bgutil_baseurl': ['http://127.0.0.1:4416'], +> }, +> 'youtubepot-bgutilhttp': { +> 'baseurl': ['http://127.0.0.1:4416'], +> }, +> }, +> # ... all the other yt_dlp settings +> } +> ``` + +Most users won't want to do that much work, so the environment variables will make it simpler to get everything working together quickly. + +> [!NOTE] +> You will need to add a new container so that your `TubeSync` container can access it: +> +> ```sh +> $ docker run --name bgutil-ytdlp-pot-provider -d \ +> -p 4416:4416 --restart unless-stopped --pull always \ +> brainicism/bgutil-ytdlp-pot-provider +> ``` + +After that is running, you can set the new environment variables to use the web services provided by that container: + +```sh +$ TUBESYNC_POT_PORT=4416 +$ export TUBESYNC_POT_PORT +$ TUBESYNC_POT_IPADDR=[Whatever IP you are using] +$ export TUBESYNC_POT_IPADDR +``` + +> [!IMPORTANT] +> Don't forget to add `-e TUBESYNC_POT_IPADDR -e TUBESYNC_POT_PORT` to the `docker run` command that you are using with `TubeSync` as well. + From ec1207b5e2953bc4e26816fd4bc3274f2881dd1c Mon Sep 17 00:00:00 2001 From: tcely Date: Wed, 11 Jun 2025 04:13:46 -0400 Subject: [PATCH 2/4] Update README.md --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index c5adc5c5..ac81f59d 100644 --- a/README.md +++ b/README.md @@ -259,6 +259,7 @@ and less common features: * [Sync or create missing metadata files](https://github.com/meeb/tubesync/blob/main/docs/create-missing-metadata.md) * [Reset tasks from the command line](https://github.com/meeb/tubesync/blob/main/docs/reset-tasks.md) * [Using PostgreSQL, MySQL or MariaDB as database backends](https://github.com/meeb/tubesync/blob/main/docs/other-database-backends.md) + * [YouTube Proof-of-Origin Tokens](https://github.com/meeb/tubesync/blob/main/docs/youtube-pot.md) * [Using cookies](https://github.com/meeb/tubesync/blob/main/docs/using-cookies.md) * [Reset metadata](https://github.com/meeb/tubesync/blob/main/docs/reset-metadata.md) From a818e59f13a6ef76491a5574b0b15b86998b5a8f Mon Sep 17 00:00:00 2001 From: tcely Date: Wed, 11 Jun 2025 05:30:34 -0400 Subject: [PATCH 3/4] Update youtube-pot.md --- docs/youtube-pot.md | 97 ++++++++++++++++++++++++++++++++++++++------- 1 file changed, 82 insertions(+), 15 deletions(-) diff --git a/docs/youtube-pot.md b/docs/youtube-pot.md index ee6b17a5..eeecaf1c 100644 --- a/docs/youtube-pot.md +++ b/docs/youtube-pot.md @@ -1,17 +1,30 @@ -After #1005 was merged, you are welcome to try the new support for the plugin. +## General information about tokens -The implementation mostly went as I planned, except for needing to use an IP address. +- https://github.com/yt-dlp/yt-dlp/wiki/PO-Token-Guide + +## YouTube Proof-of-Origin Token plugin + +To generate tokens TubeSync is using a plugin for `yt-dlp` from: +* https://github.com/Brainicism/bgutil-ytdlp-pot-provider/tree/1.1.0/plugin/yt_dlp_plugins/extractor/ + +#### Addition of plugin support + +Support for using the plugin was added in: https://github.com/meeb/tubesync/pull/1005 + +#### Plugin web server URL + +This plugin communicates with a web service to retrieve tokens, for which it uses a default URL of: `http://127.0.0.1:4416` + +Because for TubeSync `openresty` or `nginx` are what most users will connect to with their web browser, the [configuration](../config/root/etc/nginx/token_server.conf) to listen on port `4416` has been added. + +If you are using another web server instead, you should configure similar proxying of the requests, or configure a different URL for the plugin to use. > [!TIP] -> You can skip all of this `nginx` mess entirely by configuring the base URL the plugin tries to access in the settings file. +> You can set the base URL that the plugin tries to access with a custom `local_settings.py` file. > > ```python > YOUTUBE_DEFAULTS = { > 'extractor_args': { -> # old plugin versions looked under 'youtube' -> 'youtube': { -> 'getpot_bgutil_baseurl': ['http://127.0.0.1:4416'], -> }, > 'youtubepot-bgutilhttp': { > 'baseurl': ['http://127.0.0.1:4416'], > }, @@ -20,16 +33,24 @@ The implementation mostly went as I planned, except for needing to use an IP add > } > ``` -Most users won't want to do that much work, so the environment variables will make it simpler to get everything working together quickly. +#### Running the web service container the plugin communicates with + +Docker image: `brainicism/bgutil-ytdlp-pot-provider:1.1.0` + +```sh +$ docker run -d \ + --name bgutil-ytdlp-pot-service \ + -p 4416:4416 \ + --restart unless-stopped \ + brainicism/bgutil-ytdlp-pot-provider:1.1.0 +``` + +#### Configure the plugin using environment variables + +Because most users do not already have a custom `local_settings.py` file, and are not using a custom web server, an easier way to configure the plugin was developed. > [!NOTE] -> You will need to add a new container so that your `TubeSync` container can access it: -> -> ```sh -> $ docker run --name bgutil-ytdlp-pot-provider -d \ -> -p 4416:4416 --restart unless-stopped --pull always \ -> brainicism/bgutil-ytdlp-pot-provider -> ``` +> You will need to add a new container so that your `TubeSync` container can access it. After that is running, you can set the new environment variables to use the web services provided by that container: @@ -43,3 +64,49 @@ $ export TUBESYNC_POT_IPADDR > [!IMPORTANT] > Don't forget to add `-e TUBESYNC_POT_IPADDR -e TUBESYNC_POT_PORT` to the `docker run` command that you are using with `TubeSync` as well. +> [!TIP] +> Setting `TUBESYNC_POT_HTTPS` to `True` will connect using `https://`[^1] instead of `http://`. + +[^1]: Setting up valid certificates for the web service is outside the scope of this guide. + +#### Using the `--link` flag with `docker run` + +When you have both containers on the same `docker` host, it can be very convenient to use this method. + +Environment variables are automatically created by `docker`, and support for using those was included. + +For a web service container named `bgutil-ytdlp-pot-service` you would add this `--link` option to your `docker run` command for TubeSync: + +```sh +$ docker run --link 'bgutil-ytdlp-pot-service:POTServer' # everything else +``` + +#### Checking that things are working + +The web service provides a `/ping` endpoint. + +```sh +$ no_proxy='127.0.0.1' curl 'http://127.0.0.1:4416/ping' ; echo +{"token_ttl_hours":6,"server_uptime":1633.034876421,"version":"0.8.4"} +``` + +If the `curl` command above works from inside the TubeSync container, then the web server configuration is proxying the requests as expected. + +When it is not configured, the same `/ping` request receives a HTTP 502 error. + +#### More information about the web service the plugin uses + +Server: +* [Dockerfile](https://github.com/Brainicism/bgutil-ytdlp-pot-provider/blob/master/server/Dockerfile) +* [main](https://github.com/Brainicism/bgutil-ytdlp-pot-provider/blob/master/server/src/main.ts) + + +#### All of the environment variables available + +Added environment variables: + +- TUBESYNC_POT_IPADDR +- TUBESYNC_POT_PORT +- TUBESYNC_POT_HTTPS (use https:// when set) +- YT_POT_BGUTIL_BASE_URL (`youtubepot-bgutilhttp:base_url`) (not set by the user) + From 46e0480b55fa13971a9793b8f920b3c1a4ce2f1c Mon Sep 17 00:00:00 2001 From: tcely Date: Wed, 11 Jun 2025 05:44:58 -0400 Subject: [PATCH 4/4] Pin the plugin version --- Pipfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Pipfile b/Pipfile index 49b5127b..5bc0d10a 100644 --- a/Pipfile +++ b/Pipfile @@ -24,4 +24,4 @@ yt-dlp = {extras = ["default", "curl-cffi"], version = "*"} emoji = "*" brotli = "*" html5lib = "*" -bgutil-ytdlp-pot-provider = "*" +bgutil-ytdlp-pot-provider = "~=1.0"