Commit Graph

101 Commits

Author SHA1 Message Date
lwthiker
af38d2b626 Merge pull request #35 from lwthiker/chrome_android
Add signature for Chrome 99 on Android
v0.3.2
2022-03-24 21:23:22 +02:00
lwthiker
584ff54653 Add signature for Chrome 99 on Android phone
Add support for impersonating Chrome 99 on Android 12 (Pixel 6 was
chosen as an arbitrary Android phone to impersonate). The TLS signature
for Chrome on Android is identical to Chrome on Windows. The difference
is in a few HTTP headers ('sec-ch-ua-mobile', 'sec-ch-ua-platform' and
'user-agent').
2022-03-24 21:01:47 +02:00
lwthiker
a3054994aa Merge pull request #29 from lwthiker/ff98
Impersonate Firefox 98
2022-03-12 10:57:37 +02:00
lwthiker
768e37d194 Impersonate Firefox 98
Add support for impersonating Firefox 98 released a few days ago. It has
the same TLS signature as Firefox 95 so the adaptation includes changing
the user-agent only. Upgrade the NSS version used to 3.75, even though
it's not strictly necessary.
2022-03-12 10:44:24 +02:00
lwthiker
0952bca3de Merge pull request #28 from lwthiker/alpine
Add Alpine Linux images and push automatically to DockerHub
v0.3.1
2022-03-10 14:44:33 +02:00
lwthiker
aba67a4cbe Fix typo and link in README.md v0.3.1-alpha.3 2022-03-10 13:38:37 +02:00
lwthiker
2cf96e9337 Update README.md with Alpine-based Docker images 2022-03-10 13:34:56 +02:00
lwthiker
ebc8032615 Add more tags to alpine-based Docker Hub images 2022-03-10 13:23:12 +02:00
lwthiker
4ea8f859eb Install nss in final alpine image
Add 'apk add nss' for the final alpine image of the Firefox version of
curl-impersonate. This is needed for curl to find the list of root
certificates.
2022-03-10 13:17:31 +02:00
lwthiker
2e738cd7d0 Push the Alpine image for Firefox version as well v0.3.1-alpha.2 2022-03-10 12:50:17 +02:00
lwthiker
21af00356e Add action to push alpine images to Docker Hub
Add a GitHub action to push alpine images with curl-impersonate to
Docker Hub whenever a new release tag is created.
v0.3.1-alpha.1
2022-03-10 12:22:27 +02:00
lwthiker
eaeb619fa7 Add Alpine Linux build system
Add Dockerfiles for building curl-impersonate on Alpine Linux.

The Dockerfile template file was modified to support Alpine Linux,
mainly by changing the dependency installation from 'apt' to 'apk'.
The resulting alpine images are small (~11mb) and will be uploaded to
Docker hub.
2022-03-10 11:57:53 +02:00
lwthiker
91c906febd Merge pull request #27 from lwthiker/template_dockerfile
Use a template for generating Dockerfiles
2022-03-10 10:24:43 +02:00
lwthiker
d1dbfc89c5 Use a template for generating Dockerfiles
Since the firefox and chrome builds are similar except for the TLS
library used, it makes sense to unify their Dockerfiles. This commit
introduces a template Dockerfile from which both the build Dockerfiles
are generated using the simple 'mustache' template system.
2022-03-09 11:01:44 +02:00
lwthiker
446ab75876 Merge pull request #25 from lwthiker/static_brotli
Compile libbrotli statically into curl-impersonate
2022-03-08 16:12:21 +02:00
lwthiker
7df69b5107 Compile libbrotli statically into curl-impersonate
Compile libbrotli statically into curl-impersonate/libcurl-impersonate
for convenience of usage outside the container.
2022-03-08 15:45:30 +02:00
lwthiker
253285a84f Update README.md
* Add a full list of supported browsers.
* Split "Usage" to "Basic usage" and "Advanced usage"
v0.3.0
2022-03-05 14:16:11 +02:00
lwthiker
ce95990620 Split GitHub Action cache to two directories
The GitHub Action workflow uses a cache to speed up the Docker builds.
However, it seems that it only partially works, saving only the Docker
layers for the Firefox build. This is an attempt to solve it by using
two separate cache paths, one for each build.
2022-03-05 13:30:14 +02:00
lwthiker
8468747057 Merge pull request #19 from lwthiker/fix_wrapper_scripts
Fix bug in wrapper scripts parameter passing
2022-03-05 13:17:46 +02:00
lwthiker
6e9b194dd2 Merge pull request #23 from lwthiker/chrome99
Impersonate Chrome 99 and Edge 99
2022-03-05 13:16:01 +02:00
lwthiker
e554958c75 Impersonate Chrome 99 and Edge 99
Add impersonation support for Chrome 99 and Edge 99 which were just released.
Their TLS signature is identical to the previous versions (98).
The only difference is in the user agents.

curl patch from
ca13947f00
2022-03-05 13:03:27 +02:00
lwthiker
a304c61522 Run CI on push to 'main' branch as well 2022-03-05 12:17:41 +02:00
lwthiker
f11a872608 Merge pull request #21 from lwthiker/test_http_headers
Test curl-impersonate's HTTP headers
2022-03-05 12:16:46 +02:00
lwthiker
13e57934d1 Merge branch 'main' into test_http_headers 2022-03-05 12:01:28 +02:00
lwthiker
fc56ed619d Merge pull request #22 from lwthiker/github_actions
Add GitHub actions workflow to build and test curl-impersonate
2022-03-05 11:59:16 +02:00
lwthiker
d577ee92f1 Add GitHub actions badge to README.md 2022-03-05 11:11:40 +02:00
lwthiker
0771c55799 Add Docker caching to GitHub actions workflow
Add layer caching to speed up the CI builds. In addition, optimize the
Dockerfiles by moving all the dependency installations (which are
unlikely to change much) to the initial layers.
2022-03-05 10:48:53 +02:00
lwthiker
867f23fb58 Use local Docker registry in GitHub Action
The tests container copies files from the build containers. In order for
this to work in the GitHub actions workflow, the build images need to be
uploaded to a local Docker registry first.
2022-03-05 10:20:07 +02:00
lwthiker
f6554b909e Add "load: true" to GitHub actions docker steps
Make the docker images in each step available to the next steps
in the GitHub workflow by specifying "load: true".
2022-03-04 19:24:06 +02:00
lwthiker
e13a644559 Add GitHub action steps to build and run the tests 2022-03-04 18:57:13 +02:00
lwthiker
be84f39250 Add checkout step to GitHub actions workflow 2022-03-04 18:12:46 +02:00
lwthiker
635f2ca63c Add GitHub actions workflow to build and test 2022-03-04 18:10:08 +02:00
lwthiker
9bab04c170 Test curl-impersonate's HTTP headers
Add tests to verify that the HTTP headers and HTTP/2 pseudo-headers
generated by curl-impersonate match the expected ones from the browser.

The test uses a local nghttpd HTTP/2 server instance with a self-signed
certificate.
2022-03-04 17:31:32 +02:00
lwthiker
cf5c661c5a Merge pull request #20 from lwthiker/headers_fix
Fix Edge and Firefox HTTP headers
2022-03-04 17:31:19 +02:00
lwthiker
6e0e9b90da Fix Edge and Firefox HTTP headers
The HTTP headers for Edge and Firefox were slightly incorrect.

curl patches from
ee0a67058f
and
351dba44d0
2022-03-04 17:27:08 +02:00
lwthiker
044b9dcc09 Merge pull request #17 from lwthiker/safari
Impersonate Safari 15.3
2022-03-04 17:24:53 +02:00
lwthiker
ac492a9434 Fix parameter passing in Safari wrapper script 2022-03-04 17:19:09 +02:00
lwthiker
2f0c0af621 Fix bug in wrapper scripts parameter passing
Parameters to the wrapper scripts were passed incorrectly to
the 'curl-impersonate' binary.
2022-03-04 17:11:42 +02:00
lwthiker
5682b021d6 Add Safari impersonation support to libcurl
Safari can now be imperonsated with libcurl using
curl_easy_impersonate() with the "safari15_3" target or by setting the
env var CURL_IMPERSONATE to "safari15_3".

curl patch from
0340cd8b3e
2022-03-03 16:57:26 +02:00
lwthiker
f9afe9cf63 Add wrapper script to impersonate Safari 15.3
Add a new wrapper script that impersonates Safari 15.3 on MacOS 11.6.4
("Big Sur"). The wrapper script uses command line arguments
previously added to "curl-impersonate" for that purpose:
* --curves
* --signature-hashes
* --no-tls-session-ticket
* --http2-pseudo-headers-order
2022-03-03 16:16:13 +02:00
lwthiker
8bc8019152 Small refactor to the testing code 2022-03-03 15:29:41 +02:00
lwthiker
2ea7458239 Add control over HTTP/2 pseudo-headers order
Add the ability to control the order of the HTTP/2 pseudo-headers. Each
browser uses a different order for the ":method", ":authority",
":scheme" and ":path" pseudo-headers. It is therefore desirable to be
able to control it. The CURLOPT_HTTP2_PSEUDO_HEADERS_ORDER libcurl
option and '--http2-pseudo-headers-order' command line option now allow
doing that.

Patch from
dd4b76241e
2022-03-03 15:25:23 +02:00
lwthiker
979750a072 Merge pull request #16 from lwthiker/tls_options
Make some of the TLS options configurable
2022-03-03 15:20:02 +02:00
lwthiker
e897975660 Update README.md 2022-03-03 15:16:27 +02:00
lwthiker
c1aa0bb50c Make TLS session ticket extension configurable
A previous commit has enabled the TLS session ticket extension (by
removing SSL_OP_NO_TICKET) because Chrome uses it. This commit makes it
configurable via the CURLOPT_SSL_ENABLE_TICKET libcurl option or the
'--tls-session-ticker' command line flag. The goal is to impersonate
Safari which, as of version 15.3, does not use TLS session tickets.
2022-03-03 12:16:30 +02:00
lwthiker
bafc1416e3 Add full support for TLS certificate compression
This commit adds:
* Support for configuring the TLS certificate compression algorithms the
client is willing to receive via the CURLOPT_SSL_CERT_COMPRESSION option or
the '--cert-compression' command line flag.
* Support for decompressing zlib-compressed certificates in addition to
brotli.

Previously brotli decompression only was available and it was hardcoded
into the binary.
2022-03-02 15:23:24 +02:00
lwthiker
87ed6a2792 Make some of the TLS options configurable
This commit makes some of the TLS options that are used for
impersonation configurable via libcurl options and command line flags to
curl-impersonate. The goal is to give more flexibility in configuring
the TLS extensions instead of hardcoding everything into the binary.
This will enable using the same binary for impersonating different
browsers (e.g. Safari).

The following options are now present:
* CURLOPT_SSL_EC_CURVES and the '--curves' flag are now usable. These
were present in the upstream curl but only for OpenSSL builds. This
commit also enables them for BoringSSL. They configure TLS extension
'supported_groups' (no. 10).
* CURLOPT_SSL_ENABLE_NPN and the '--no-npn' flags are usable. These were
present in the upstream curl but were disabled in a previous commit by
commenting out the relevant code (as Chrome disables NPN). They now work
and the wrapper scripts use the '--no-npn' flag.
* CURLOPT_SSL_ENABLE_ALPS and the '--alps' flag were added. These
control the ALPS TLS extension that Chrome uses.
* CURLOPT_SSL_SIG_HASH_ALGS and the '--signature-hashes' option were
added. These control the clien't list of supported signature & hash
algorithms, i.e. TLS extension 'signature_algorithms' (no. 13).
2022-03-02 10:28:35 +02:00
lwthiker
051ccfd5e6 Merge pull request #15 from lwthiker/boringssl_old_ciphers
Restore old ciphers in BoringSSL
2022-03-02 09:33:36 +02:00
lwthiker
ce36fa86a1 Merge pull request #13 from lwthiker/libcurl-impersonate
Add impersonation support to libcurl
2022-02-28 10:25:27 +02:00
lwthiker
a7cbfd9fed Add libcurl impersonation support in Firefox build
48415a4b00 added impersonation
capabilities to libcurl in the Chrome build. This adds the same
capabilities to the Firefox build as well.

curl-impersonate.patch generated from
b30b245b72
2022-02-28 10:18:04 +02:00