Commit Graph

58 Commits

Author SHA1 Message Date
bashonly
e98695549e [rh:curl_cffi] Support curl_cffi 0.11.x, 0.12.x, 0.13.x (#13989)
Authored by: bashonly
2025-08-11 03:16:07 +00:00
bashonly
8175f3738f [rh:requests] Bump minimum required version of urllib3 to 2.0.2 (#13939)
- urllib3 1.26.x gives unexpected results with partial reads: https://github.com/urllib3/urllib3/issues/2128
- urllib3 2.0.0 and 2.0.1 were yanked from PyPI: https://github.com/urllib3/urllib3/issues/3009

Closes #13927
Authored by: bashonly
2025-08-06 19:00:53 +00:00
bashonly
a4561c7a66 [rh:requests] Refactor default headers (#13785)
Authored by: bashonly
2025-07-20 23:20:58 +00:00
Simon Sawicki
c316416b97 [rh:requests] Do not allocate 2GB on read (#13603)
Fixes c2ff2dbaec

Authored by: Grub4K
2025-07-02 01:42:00 +02:00
Simon Sawicki
c2ff2dbaec [rh:requests] Work around partial read dropping data (#13599)
Authored by: Grub4K
2025-07-02 00:12:43 +02:00
coletdjnz
2685654a37 [ie/youtube] Add a PO Token Provider Framework (#12840)
Some checks failed
CodeQL / Analyze (python) (push) Has been cancelled
Core Tests / Core Tests (ubuntu-latest, 3.10) (push) Has been cancelled
Core Tests / Core Tests (ubuntu-latest, 3.11) (push) Has been cancelled
Core Tests / Core Tests (ubuntu-latest, 3.12) (push) Has been cancelled
Core Tests / Core Tests (ubuntu-latest, 3.13) (push) Has been cancelled
Core Tests / Core Tests (ubuntu-latest, pypy-3.10) (push) Has been cancelled
Core Tests / Core Tests (windows-latest, 3.10) (push) Has been cancelled
Core Tests / Core Tests (windows-latest, 3.12) (push) Has been cancelled
Core Tests / Core Tests (windows-latest, 3.13) (push) Has been cancelled
Core Tests / Core Tests (windows-latest, 3.9) (push) Has been cancelled
Core Tests / Core Tests (windows-latest, pypy-3.10) (push) Has been cancelled
Download Tests / Quick Download Tests (push) Has been cancelled
Download Tests / Full Download Tests (ubuntu-latest, 3.10) (push) Has been cancelled
Download Tests / Full Download Tests (ubuntu-latest, 3.11) (push) Has been cancelled
Download Tests / Full Download Tests (ubuntu-latest, 3.12) (push) Has been cancelled
Download Tests / Full Download Tests (ubuntu-latest, 3.13) (push) Has been cancelled
Download Tests / Full Download Tests (ubuntu-latest, pypy-3.10) (push) Has been cancelled
Download Tests / Full Download Tests (windows-latest, 3.9) (push) Has been cancelled
Download Tests / Full Download Tests (windows-latest, pypy-3.10) (push) Has been cancelled
Quick Test / Core Test (push) Has been cancelled
Quick Test / Code check (push) Has been cancelled
Release (master) / release (push) Has been cancelled
Release (master) / publish_pypi (push) Has been cancelled
https://github.com/yt-dlp/yt-dlp/tree/master/yt_dlp/extractor/youtube/pot/README.md

Authored by: coletdjnz
2025-05-18 13:45:26 +12:00
doe1080
ceab4d5ed6 [networking] Add PATCH request shortcut (#12884)
Some checks failed
Download Tests / Quick Download Tests (push) Waiting to run
Download Tests / Full Download Tests (ubuntu-latest, 3.10) (push) Waiting to run
Download Tests / Full Download Tests (ubuntu-latest, 3.11) (push) Waiting to run
Download Tests / Full Download Tests (ubuntu-latest, 3.12) (push) Waiting to run
Download Tests / Full Download Tests (ubuntu-latest, 3.13) (push) Waiting to run
Download Tests / Full Download Tests (ubuntu-latest, pypy-3.10) (push) Waiting to run
Download Tests / Full Download Tests (windows-latest, 3.9) (push) Waiting to run
Download Tests / Full Download Tests (windows-latest, pypy-3.10) (push) Waiting to run
Quick Test / Core Test (push) Waiting to run
Quick Test / Code check (push) Waiting to run
Release (master) / release (push) Waiting to run
Release (master) / publish_pypi (push) Blocked by required conditions
Core Tests / Core Tests (ubuntu-latest, 3.10) (push) Has been cancelled
Core Tests / Core Tests (ubuntu-latest, 3.11) (push) Has been cancelled
Core Tests / Core Tests (ubuntu-latest, 3.12) (push) Has been cancelled
Core Tests / Core Tests (ubuntu-latest, 3.13) (push) Has been cancelled
Core Tests / Core Tests (ubuntu-latest, pypy-3.10) (push) Has been cancelled
Core Tests / Core Tests (windows-latest, 3.10) (push) Has been cancelled
Core Tests / Core Tests (windows-latest, 3.12) (push) Has been cancelled
Core Tests / Core Tests (windows-latest, 3.13) (push) Has been cancelled
Core Tests / Core Tests (windows-latest, 3.9) (push) Has been cancelled
Core Tests / Core Tests (windows-latest, pypy-3.10) (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
Authored by: doe1080
2025-04-18 11:46:19 +12:00
Simon Sawicki
9bf23902ce [rh:curl_cffi] Support curl_cffi 0.10.x (#12670)
Some checks failed
CodeQL / Analyze (python) (push) Waiting to run
Download Tests / Quick Download Tests (push) Waiting to run
Download Tests / Full Download Tests (ubuntu-latest, 3.10) (push) Waiting to run
Download Tests / Full Download Tests (ubuntu-latest, 3.11) (push) Waiting to run
Download Tests / Full Download Tests (ubuntu-latest, 3.12) (push) Waiting to run
Download Tests / Full Download Tests (ubuntu-latest, 3.13) (push) Waiting to run
Download Tests / Full Download Tests (ubuntu-latest, pypy-3.10) (push) Waiting to run
Download Tests / Full Download Tests (windows-latest, 3.9) (push) Waiting to run
Download Tests / Full Download Tests (windows-latest, pypy-3.10) (push) Waiting to run
Quick Test / Core Test (push) Waiting to run
Quick Test / Code check (push) Waiting to run
Release (master) / release (push) Waiting to run
Release (master) / publish_pypi (push) Blocked by required conditions
Core Tests / Core Tests (ubuntu-latest, 3.10) (push) Has been cancelled
Core Tests / Core Tests (ubuntu-latest, 3.11) (push) Has been cancelled
Core Tests / Core Tests (ubuntu-latest, 3.12) (push) Has been cancelled
Core Tests / Core Tests (ubuntu-latest, 3.13) (push) Has been cancelled
Core Tests / Core Tests (ubuntu-latest, pypy-3.10) (push) Has been cancelled
Core Tests / Core Tests (windows-latest, 3.10) (push) Has been cancelled
Core Tests / Core Tests (windows-latest, 3.12) (push) Has been cancelled
Core Tests / Core Tests (windows-latest, 3.13) (push) Has been cancelled
Core Tests / Core Tests (windows-latest, 3.9) (push) Has been cancelled
Core Tests / Core Tests (windows-latest, pypy-3.10) (push) Has been cancelled
Authored by: Grub4K
2025-03-23 00:15:20 +01:00
Simon Sawicki
95f8df2f79 [networking] Always add unsupported suffix on version mismatch (#12626)
Authored by: Grub4K
2025-03-16 12:45:44 +01:00
Simon Sawicki
7d18fed8f1 [networking] Add keep_header_casing extension (#11652)
Some checks failed
CodeQL / Analyze (python) (push) Waiting to run
Download Tests / Quick Download Tests (push) Waiting to run
Download Tests / Full Download Tests (ubuntu-latest, 3.10) (push) Waiting to run
Download Tests / Full Download Tests (ubuntu-latest, 3.11) (push) Waiting to run
Download Tests / Full Download Tests (ubuntu-latest, 3.12) (push) Waiting to run
Download Tests / Full Download Tests (ubuntu-latest, 3.13) (push) Waiting to run
Download Tests / Full Download Tests (ubuntu-latest, pypy-3.10) (push) Waiting to run
Download Tests / Full Download Tests (windows-latest, 3.9) (push) Waiting to run
Download Tests / Full Download Tests (windows-latest, pypy-3.10) (push) Waiting to run
Quick Test / Core Test (push) Waiting to run
Quick Test / Code check (push) Waiting to run
Release (master) / release (push) Waiting to run
Release (master) / publish_pypi (push) Blocked by required conditions
Core Tests / Core Tests (ubuntu-latest, 3.10) (push) Has been cancelled
Core Tests / Core Tests (ubuntu-latest, 3.11) (push) Has been cancelled
Core Tests / Core Tests (ubuntu-latest, 3.12) (push) Has been cancelled
Core Tests / Core Tests (ubuntu-latest, 3.13) (push) Has been cancelled
Core Tests / Core Tests (ubuntu-latest, pypy-3.10) (push) Has been cancelled
Core Tests / Core Tests (windows-latest, 3.10) (push) Has been cancelled
Core Tests / Core Tests (windows-latest, 3.12) (push) Has been cancelled
Core Tests / Core Tests (windows-latest, 3.13) (push) Has been cancelled
Core Tests / Core Tests (windows-latest, 3.9) (push) Has been cancelled
Core Tests / Core Tests (windows-latest, pypy-3.10) (push) Has been cancelled
Authored by: coletdjnz, Grub4K

Co-authored-by: coletdjnz <coletdjnz@protonmail.com>
2025-03-03 00:10:01 +01:00
bashonly
c6387abc1a [cleanup] Misc (#10807)
Closes #10751, Closes #10769, Closes #10791
Authored by: bashonly, Codenade, pzhlkj6612, seproDev, coletdjnz, grqz, Grub4K

Co-authored-by: Codenade <amadeus.dorian04@gmail.com>
Co-authored-by: Mozi <29089388+pzhlkj6612@users.noreply.github.com>
Co-authored-by: sepro <4618135+seproDev@users.noreply.github.com>
Co-authored-by: coletdjnz <coletdjnz@protonmail.com>
Co-authored-by: N/Ame <173015200+grqz@users.noreply.github.com>
Co-authored-by: Simon Sawicki <contact@grub4k.xyz>
2024-09-27 22:46:22 +00:00
bashonly
5bb1aa04da [networking] Pin curl-cffi version to < 0.7.2 (#11092)
Ref: https://github.com/lexiforest/curl_cffi/issues/394

Authored by: bashonly
2024-09-25 20:59:20 +00:00
coletdjnz
d1c4d88b2d [networking] Fix handler not being added to RequestError (#10955)
Some checks failed
CodeQL / Analyze (python) (push) Has been cancelled
Core Tests / Core Tests (ubuntu-latest, 3.10) (push) Has been cancelled
Core Tests / Core Tests (ubuntu-latest, 3.11) (push) Has been cancelled
Core Tests / Core Tests (ubuntu-latest, 3.12) (push) Has been cancelled
Core Tests / Core Tests (ubuntu-latest, 3.9) (push) Has been cancelled
Core Tests / Core Tests (ubuntu-latest, pypy-3.10) (push) Has been cancelled
Core Tests / Core Tests (ubuntu-latest, pypy-3.8) (push) Has been cancelled
Core Tests / Core Tests (windows-latest, 3.12) (push) Has been cancelled
Core Tests / Core Tests (windows-latest, 3.8) (push) Has been cancelled
Core Tests / Core Tests (windows-latest, pypy-3.9) (push) Has been cancelled
Download Tests / Quick Download Tests (push) Has been cancelled
Download Tests / Full Download Tests (ubuntu-latest, 3.10) (push) Has been cancelled
Download Tests / Full Download Tests (ubuntu-latest, 3.11) (push) Has been cancelled
Download Tests / Full Download Tests (ubuntu-latest, 3.12) (push) Has been cancelled
Download Tests / Full Download Tests (ubuntu-latest, pypy-3.10) (push) Has been cancelled
Download Tests / Full Download Tests (ubuntu-latest, pypy-3.8) (push) Has been cancelled
Download Tests / Full Download Tests (windows-latest, 3.8) (push) Has been cancelled
Download Tests / Full Download Tests (windows-latest, pypy-3.9) (push) Has been cancelled
Quick Test / Core Test (push) Has been cancelled
Quick Test / Code check (push) Has been cancelled
Release (master) / release (push) Has been cancelled
Authored by: coletdjnz
2024-09-08 19:32:44 +12:00
coletdjnz
6f9e653743 [rh:websockets] Upgrade websockets to 13.0 (#10815)
Some checks failed
CodeQL / Analyze (python) (push) Has been cancelled
Core Tests / Core Tests (ubuntu-latest, 3.10) (push) Has been cancelled
Core Tests / Core Tests (ubuntu-latest, 3.11) (push) Has been cancelled
Core Tests / Core Tests (ubuntu-latest, 3.12) (push) Has been cancelled
Core Tests / Core Tests (ubuntu-latest, 3.9) (push) Has been cancelled
Core Tests / Core Tests (ubuntu-latest, pypy-3.10) (push) Has been cancelled
Core Tests / Core Tests (ubuntu-latest, pypy-3.8) (push) Has been cancelled
Core Tests / Core Tests (windows-latest, 3.12) (push) Has been cancelled
Core Tests / Core Tests (windows-latest, 3.8) (push) Has been cancelled
Core Tests / Core Tests (windows-latest, pypy-3.9) (push) Has been cancelled
Download Tests / Quick Download Tests (push) Has been cancelled
Download Tests / Full Download Tests (ubuntu-latest, 3.10) (push) Has been cancelled
Download Tests / Full Download Tests (ubuntu-latest, 3.11) (push) Has been cancelled
Download Tests / Full Download Tests (ubuntu-latest, 3.12) (push) Has been cancelled
Download Tests / Full Download Tests (ubuntu-latest, pypy-3.10) (push) Has been cancelled
Download Tests / Full Download Tests (ubuntu-latest, pypy-3.8) (push) Has been cancelled
Download Tests / Full Download Tests (windows-latest, 3.8) (push) Has been cancelled
Download Tests / Full Download Tests (windows-latest, pypy-3.9) (push) Has been cancelled
Quick Test / Core Test (push) Has been cancelled
Quick Test / Code check (push) Has been cancelled
Release (master) / release (push) Has been cancelled
Fixes CI hanging

Authored by: coletdjnz
2024-08-21 19:17:26 +12:00
coletdjnz
150ecc45d9 [networking] Add legacy_ssl request extension (#10448)
Supported by Urllib, Requests and Websockets request handlers. Ignored by CurlCFFI.

Also added couple cookie-related tests.

Authored by: coletdjnz
2024-07-14 11:22:43 +12:00
coletdjnz
42bfca00a6 [rh:curl_cffi] Support curl_cffi 0.7.X
Authored by: coletdjnz
2024-07-09 18:36:15 -05:00
bashonly
6aaf96a3d6 [cleanup] Misc (#10075)
Closes #10303
Authored by: bashonly, seproDev, jucor, c-basalt

Co-authored-by: sepro <4618135+seproDev@users.noreply.github.com>
Co-authored-by: Julien Cornebise <julien@cornebise.com>
Co-authored-by: c-basalt <117849907+c-basalt@users.noreply.github.com>
2024-07-01 22:51:27 +00:00
sepro
7814c50948 [cleanup] Bump ruff to 0.5.x (#10282)
Authored by: seproDev
2024-06-29 17:30:57 +02:00
sepro
add96eb9f8 [cleanup] Add more ruff rules (#10149)
Authored by: seproDev

Reviewed-by: bashonly <88596187+bashonly@users.noreply.github.com>
Reviewed-by: Simon Sawicki <contact@grub4k.xyz>
2024-06-12 01:09:58 +02:00
bashonly
db50f19d76 [rh:requests] Bump minimum requests version to 2.32.2 (#10079)
Closes #10078
Authored by: bashonly
2024-06-01 18:57:23 +00:00
Simon Sawicki
3f7999533e [rh:requests] Patch support for requests 2.32.2+ (#9992)
Authored by: Grub4K
2024-05-22 16:22:25 +02:00
coletdjnz
c36513f1be [rh:requests] Update to requests 2.32.0 (#9980)
Authored by: coletdjnz
2024-05-20 21:44:41 +00:00
coletdjnz
53b4d44f55 [test] Fix connect timeout test (#9906)
Fixes https://github.com/yt-dlp/yt-dlp/issues/9659

Authored by: coletdjnz
2024-05-18 19:12:21 +12:00
coletdjnz
3c7a287e28 [test] Add HTTP proxy tests (#9578)
Also fixes HTTPS proxies for curl_cffi

Authored by: coletdjnz
2024-05-11 10:06:58 +12:00
bashonly
bec9a59e8e [networking] Add extensions attribute to Response (#9756)
CurlCFFIRH now provides an `impersonate` field in its responses' extensions

Authored by: bashonly
2024-05-04 22:19:42 +00:00
bashonly
216f6a3cb5 [cleanup] Misc (#9426)
Authored by: bashonly, pukkandan
2024-04-09 16:12:26 +00:00
luiso1979
79a451e576 [networking] Respect SSLKEYLOGFILE environment variable (#9543)
Authored by: luiso1979
2024-04-08 21:53:30 +02:00
bashonly
50c2935231 [ie] Add extractor impersonate API (#9474)
Authored by: bashonly, Grub4K, pukkandan
2024-03-30 23:18:07 +00:00
coletdjnz
e5d4f11104 [rh:websockets] Workaround race condition causing issues on PyPy (#9514)
Authored by: coletdjnz
2024-03-23 11:27:10 +13:00
coletdjnz
52f5be1f1e [rh:curlcffi] Add support for curl_cffi
Authored by: coletdjnz, Grub4K, pukkandan, bashonly

Co-authored-by: Simon Sawicki <contact@grub4k.xyz>
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
Co-authored-by: bashonly <bashonly@protonmail.com>
2024-03-16 23:15:11 -05:00
coletdjnz
0b81d4d252 Add new options --impersonate and --list-impersonate-targets
Authored by: coletdjnz, Grub4K, pukkandan, bashonly

Co-authored-by: Simon Sawicki <contact@grub4k.xyz>
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
Co-authored-by: bashonly <bashonly@protonmail.com>
2024-03-16 23:14:13 -05:00
pukkandan
93240fc184 [cleanup] Fix misc bugs (#8968)
Closes #8816

Authored by: bashonly, seproDev, pukkandan, Grub4k
2024-03-11 00:51:26 +05:30
pukkandan
47ab66db0f [docs] Misc Cleanup (#8977)
Closes #8355, #8944

Authored by: bashonly, Grub4k, Arthurszzz, seproDev, pukkandan

Co-authored-by: sepro <4618135+seproDev@users.noreply.github.com>
Co-authored-by: bashonly <bashonly@protonmail.com>
Co-authored-by: Arthurszzz <minecraftgamerarthur@gmail.com>
Co-authored-by: Simon Sawicki <accounts@grub4k.xyz>
Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
2024-03-11 00:48:47 +05:30
Simon Sawicki
dbd8b1bff9 Improve 069b2aedae
Authored by: Grub4k
2024-03-10 20:44:53 +05:30
pukkandan
069b2aedae Create ydl._request_director when needed 2024-02-25 06:06:42 +05:30
coletdjnz
0085e2bab8 [rh] Remove additional logging handlers on close (#9032)
Fixes https://github.com/yt-dlp/yt-dlp/issues/8922

Authored by: coletdjnz
2024-02-18 11:32:34 +13:00
coletdjnz
35f4f764a7 [rh:requests] Apply remove_dot_segments to absolute redirect locations
Fixes https://github.com/yt-dlp/yt-dlp/issues/9020

Authored by: coletdjnz
2024-01-21 10:03:33 +13:00
coletdjnz
811d298b23 [networking] Remove _CompatHTTPError (#8871)
Use `yt_dlp.networking.exceptions.HTTPError`.
`_CompatHTTPError` was to help with transition to the networking framework.

Authored by: coletdjnz
2024-01-20 15:26:50 +13:00
Simon Sawicki
f9fb3ce86e [cleanup] Misc (#8598)
Authored by: bashonly, pukkandan, seproDev, Grub4K

Co-authored-by: bashonly <bashonly@protonmail.com>
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
Co-authored-by: sepro <4618135+seproDev@users.noreply.github.com>
2023-12-30 22:27:36 +01:00
coletdjnz
ccfd70f4c2 [rh:websockets] Migrate websockets to networking framework (#7720)
* Adds a basic WebSocket framework
* Introduces new minimum `websockets` version of 12.0
* Deprecates `WebSocketsWrapper`

Fixes https://github.com/yt-dlp/yt-dlp/issues/8439

Authored by: coletdjnz
2023-11-20 08:04:04 +00:00
Simon Sawicki
b012271d01 [cleanup] Misc (#8510)
Authored by: bashonly, coletdjnz, dirkf, gamer191, seproDev, Grub4K
2023-11-14 22:40:38 +01:00
Simon Sawicki
4e38e2ae9d [rh:requests] Handle both bytes and int for IncompleteRead.partial (Fix 8a8b54523a) (#8348)
Authored by: bashonly, coletdjnz, Grub4K
2023-10-15 10:54:38 +02:00
coletdjnz
8a8b54523a [rh:requests] Add handler for requests HTTP library (#3668)
Adds support for HTTPS proxies and persistent connections (keep-alive)

Closes https://github.com/yt-dlp/yt-dlp/issues/1890
Resolves https://github.com/yt-dlp/yt-dlp/issues/4070
Resolves https://github.com/ytdl-org/youtube-dl/issues/32549
Resolves https://github.com/ytdl-org/youtube-dl/issues/14523
Resolves https://github.com/ytdl-org/youtube-dl/issues/13734

Authored by: coletdjnz, Grub4K, bashonly
2023-10-13 23:33:00 +00:00
bashonly
5ca095cbcd [cleanup] Misc (#8182)
Closes #7796, Closes #8028
Authored by: barsnick, sqrtNOT, gamer191, coletdjnz, Grub4K, bashonly
2023-09-23 20:00:31 +00:00
coletdjnz
20fbbd9249 [networking] Fix various socks proxy bugs (#8065)
- Fixed support for IPv6 socks proxies
- Fixed support for IPv6 over socks5
- Fixed --source-address not being obeyed for socks4 and socks5
- Fixed socks4a when the destination address is an IPv4 address

Closes https://github.com/yt-dlp/yt-dlp/issues/7959
Fixes https://github.com/ytdl-org/youtube-dl/issues/15368

Authored by: coletdjnz
Co-authored-by: Simon Sawicki <accounts@grub4k.xyz>
Co-authored-by: bashonly <bashonly@bashonly.com>
2023-09-18 07:33:26 +00:00
Simon Sawicki
836e06d246 [core] Fix support for upcoming Python 3.12 (#8130)
This also adds the following test runners:
- `3.12-dev` on `ubuntu-latest`
- `3.12-dev` on `windows-latest`
- `pypy-3.10` on `ubuntu-latest`

Authored by: Grub4K
2023-09-17 12:56:50 +02:00
Simon Sawicki
77bff23ee9 Bugfix for 59e92b1f18
Closes #8012

Authored by: Grub4K
2023-09-02 15:18:04 +02:00
Simon Sawicki
59e92b1f18 [rh/urllib] Simplify gzip decoding (#7611)
Authored by: Grub4K
2023-08-27 00:13:30 +02:00
coletdjnz
db7b054a61 [networking] Add request handler preference framework (#7603)
Preference functions that take a request and a request handler instance can be registered to prioritize different request handlers per request.

Authored by: coletdjnz
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
2023-08-04 22:17:48 +00:00
pukkandan
6148833f5c [cleanup] Misc 2023-07-30 04:06:18 +05:30