diff --git a/Pipfile b/Pipfile index 1d27d783..a23c3dbb 100644 --- a/Pipfile +++ b/Pipfile @@ -25,5 +25,5 @@ yt-dlp = {extras = ["default", "curl-cffi"], version = "*"} emoji = "*" brotli = "*" html5lib = "*" -bgutil-ytdlp-pot-provider = "*" +bgutil-ytdlp-pot-provider = "~=1.0" babi = "*" diff --git a/README.md b/README.md index c5adc5c5..ac81f59d 100644 --- a/README.md +++ b/README.md @@ -259,6 +259,7 @@ and less common features: * [Sync or create missing metadata files](https://github.com/meeb/tubesync/blob/main/docs/create-missing-metadata.md) * [Reset tasks from the command line](https://github.com/meeb/tubesync/blob/main/docs/reset-tasks.md) * [Using PostgreSQL, MySQL or MariaDB as database backends](https://github.com/meeb/tubesync/blob/main/docs/other-database-backends.md) + * [YouTube Proof-of-Origin Tokens](https://github.com/meeb/tubesync/blob/main/docs/youtube-pot.md) * [Using cookies](https://github.com/meeb/tubesync/blob/main/docs/using-cookies.md) * [Reset metadata](https://github.com/meeb/tubesync/blob/main/docs/reset-metadata.md) diff --git a/docs/youtube-pot.md b/docs/youtube-pot.md new file mode 100644 index 00000000..eeecaf1c --- /dev/null +++ b/docs/youtube-pot.md @@ -0,0 +1,112 @@ +## General information about tokens + +- https://github.com/yt-dlp/yt-dlp/wiki/PO-Token-Guide + +## YouTube Proof-of-Origin Token plugin + +To generate tokens TubeSync is using a plugin for `yt-dlp` from: +* https://github.com/Brainicism/bgutil-ytdlp-pot-provider/tree/1.1.0/plugin/yt_dlp_plugins/extractor/ + +#### Addition of plugin support + +Support for using the plugin was added in: https://github.com/meeb/tubesync/pull/1005 + +#### Plugin web server URL + +This plugin communicates with a web service to retrieve tokens, for which it uses a default URL of: `http://127.0.0.1:4416` + +Because for TubeSync `openresty` or `nginx` are what most users will connect to with their web browser, the [configuration](../config/root/etc/nginx/token_server.conf) to listen on port `4416` has been added. + +If you are using another web server instead, you should configure similar proxying of the requests, or configure a different URL for the plugin to use. + +> [!TIP] +> You can set the base URL that the plugin tries to access with a custom `local_settings.py` file. +> +> ```python +> YOUTUBE_DEFAULTS = { +> 'extractor_args': { +> 'youtubepot-bgutilhttp': { +> 'baseurl': ['http://127.0.0.1:4416'], +> }, +> }, +> # ... all the other yt_dlp settings +> } +> ``` + +#### Running the web service container the plugin communicates with + +Docker image: `brainicism/bgutil-ytdlp-pot-provider:1.1.0` + +```sh +$ docker run -d \ + --name bgutil-ytdlp-pot-service \ + -p 4416:4416 \ + --restart unless-stopped \ + brainicism/bgutil-ytdlp-pot-provider:1.1.0 +``` + +#### Configure the plugin using environment variables + +Because most users do not already have a custom `local_settings.py` file, and are not using a custom web server, an easier way to configure the plugin was developed. + +> [!NOTE] +> You will need to add a new container so that your `TubeSync` container can access it. + +After that is running, you can set the new environment variables to use the web services provided by that container: + +```sh +$ TUBESYNC_POT_PORT=4416 +$ export TUBESYNC_POT_PORT +$ TUBESYNC_POT_IPADDR=[Whatever IP you are using] +$ export TUBESYNC_POT_IPADDR +``` + +> [!IMPORTANT] +> Don't forget to add `-e TUBESYNC_POT_IPADDR -e TUBESYNC_POT_PORT` to the `docker run` command that you are using with `TubeSync` as well. + +> [!TIP] +> Setting `TUBESYNC_POT_HTTPS` to `True` will connect using `https://`[^1] instead of `http://`. + +[^1]: Setting up valid certificates for the web service is outside the scope of this guide. + +#### Using the `--link` flag with `docker run` + +When you have both containers on the same `docker` host, it can be very convenient to use this method. + +Environment variables are automatically created by `docker`, and support for using those was included. + +For a web service container named `bgutil-ytdlp-pot-service` you would add this `--link` option to your `docker run` command for TubeSync: + +```sh +$ docker run --link 'bgutil-ytdlp-pot-service:POTServer' # everything else +``` + +#### Checking that things are working + +The web service provides a `/ping` endpoint. + +```sh +$ no_proxy='127.0.0.1' curl 'http://127.0.0.1:4416/ping' ; echo +{"token_ttl_hours":6,"server_uptime":1633.034876421,"version":"0.8.4"} +``` + +If the `curl` command above works from inside the TubeSync container, then the web server configuration is proxying the requests as expected. + +When it is not configured, the same `/ping` request receives a HTTP 502 error. + +#### More information about the web service the plugin uses + +Server: +* [Dockerfile](https://github.com/Brainicism/bgutil-ytdlp-pot-provider/blob/master/server/Dockerfile) +* [main](https://github.com/Brainicism/bgutil-ytdlp-pot-provider/blob/master/server/src/main.ts) + + +#### All of the environment variables available + +Added environment variables: + +- TUBESYNC_POT_IPADDR +- TUBESYNC_POT_PORT +- TUBESYNC_POT_HTTPS (use https:// when set) +- YT_POT_BGUTIL_BASE_URL (`youtubepot-bgutilhttp:base_url`) (not set by the user) +