Merge branch 'main' into libcurl-impersonate

2026-09-04 16:10:29 +00:00 · 2022-02-27 23:38:49 +02:00
parent 6dad23b4b8 b918636a5b
commit 839e4e6ea7
5 changed files with 110 additions and 11 deletions
--- a/README.md
+++ b/README.md
@@ -1,4 +1,4 @@
-A special compilation of [curl](https://github.com/curl/curl) that makes it impersonate real browsers. Currently supports Chrome & Firefox. This curl binary is able to perform a TLS handshake that is identical to that of a real browser.
+A special compilation of [curl](https://github.com/curl/curl) that makes it impersonate real browsers. Currently supports Chrome, Edge & Firefox. This curl binary is able to perform a TLS handshake that is identical to that of a real browser.

 ## Why?
 When you use an HTTP client with a TLS website, it first performs a TLS handshake. The first message of that handshake is called Client Hello. The Client Hello message that curl produces differs drastically from that of a real browser. Compare the following Wireshark capture. Left is a regular curl, right is Firefox.
@@ -30,6 +30,7 @@ docker build -t curl-impersonate-chrome chrome/
 The resulting image contains:
 * `/build/out/curl-impersonate` - The curl binary that can impersonate Chrome. It is compiled statically against libcurl, BoringSSL, and libnghttp2 so that it won't conflict with any existing libraries on your system. You can use it from the container or copy it out. Tested to work on Ubuntu 20.04.
 * `/build/out/curl_chrome98` - A wrapper script that launches `curl-impersonate` with the needed headers and ciphers to impersonate Chrome 98.
+* `/build/out/curl_edge98` - Same but with Edge 98 (which is based on Chromium).
 * `/build/out/libcurl-impersonate.so` - libcurl compiled with impersonation support. See [Usage](#usage) below for more details.

 You can use them inside the docker, copy them out using `docker cp` or use them in a multi-stage docker build. If you use it outside this container:
@@ -91,17 +92,13 @@ This repository contains two main folders:
 The layout is similar for both. For example, the Firefox directory contains:
 * [Dockerfile](firefox/Dockerfile) - Used to build `curl-impersonate` with all dependencies.
 * [curl_ff91esr](firefox/curl_ff91esr), [curl_ff95](curl_ff95) - Wrapper scripts that launch `curl-impersonate` with the correct flags.
-* [curl-lib-nss.patch](firefox/patches/curl-lib-nss.patch) - The main patch that makes curl use the same TLS extensions as Firefox.
+* [curl-impersonate.patch](firefox/patches/curl-impersonate.patch) - The main patch that makes curl use the same TLS extensions as Firefox. Also makes curl compile statically with libnghttp2 and libnss.
 * [libnghttp2-pc.patch](firefox/patches/libnghttp2-pc.patch) - Patch to make libnghttp2 compile statically.
-* [curl-configure.patch](firefox/patches/curl-configure.patch) - Patch to make curl compile with a static libnghttp2.
-* [curl-static-libnss.patch](firefox/patches/curl-static-libnss.patch) - Patch to make curl compile with a static libnss.

 Other files of interest:
 * [tests/signatures.yaml](tests/signatures.yaml) - YAML database of known browser signatures that can be impersonated.

-## What's next?
-This was done in a very hacky way, but I hope it could be turned into a real project. Imagine that you could run:
-```
-curl --impersonate ff95
-```
-and it would behave exactly like Firefox 95. It could then be wrappped with a nice Python library.
+## Contributing
+If you'd like to help, please check out the [open issues](https://github.com/lwthiker/curl-impersonate/issues). You can open a pull request with your changes.
+
+This repository contains the build process for `curl-impersonate`. The actual patches to `curl` are maintained in a [separate repository](https://github.com/lwthiker/curl) forked from the upstream curl. The changes are maintained in the [impersonate-firefox](https://github.com/lwthiker/curl/tree/impersonate-firefox)  and [impersonate-chrome](https://github.com/lwthiker/curl/tree/impersonate-chrome) branches.
--- a/chrome/Dockerfile
+++ b/chrome/Dockerfile
@@ -91,5 +91,5 @@ RUN ver=$(readlink -f curl-7.81.0/lib/.libs/libcurl.so | sed 's/.*so\.//') && \
    strip "out/libcurl-impersonate.so.$ver"

 # Wrapper scripts
-COPY curl_chrome* out/
+COPY curl_chrome* curl_edge* out/
 RUN chmod +x out/curl_*
--- a/chrome/curl_edge98
+++ b/chrome/curl_edge98
@@ -0,0 +1,24 @@
+#!/bin/bash
+
+# Find the directory of this script
+dir=`echo "$0" | sed 's%/[^/]*$%%'`
+
+# The list of ciphers can be obtained by looking at the Client Hello message in
+# Wireshark, then converting it using this reference
+# https://wiki.mozilla.org/Security/Cipher_Suites
+"$dir/curl-impersonate" \
+    --ciphers TLS_AES_128_GCM_SHA256,TLS_AES_256_GCM_SHA384,TLS_CHACHA20_POLY1305_SHA256,ECDHE-ECDSA-AES128-GCM-SHA256,ECDHE-RSA-AES128-GCM-SHA256,ECDHE-ECDSA-AES256-GCM-SHA384,ECDHE-RSA-AES256-GCM-SHA384,ECDHE-ECDSA-CHACHA20-POLY1305,ECDHE-RSA-CHACHA20-POLY1305,ECDHE-RSA-AES128-SHA,ECDHE-RSA-AES256-SHA,AES128-GCM-SHA256,AES256-GCM-SHA384,AES128-SHA,AES256-SHA \
+    -H 'sec-ch-ua: " Not A;Brand";v="99", "Chromium";v="98", "Microsoft Edge";v="98"' \
+    -H 'sec-ch-ua-mobile: ?0' \
+    -H 'sec-ch-ua-platform: "Windows"' \
+    -H 'Upgrade-Insecure-Requests: 1' \
+    -H 'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.102 Safari/537.36 Edg/98.0.1108.62' \
+    -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9' \
+    -H 'Sec-Fetch-Site: none' \
+    -H 'Sec-Fetch-Mode: navigate' \
+    -H 'Sec-Fetch-User: ?1' \
+    -H 'Sec-Fetch-Dest: document' \
+    -H 'Accept-Encoding: gzip, deflate, br' \
+    -H 'Accept-Language: en-US,en;q=0.9' \
+    --http2 --false-start --tlsv1.2 --compressed \
+    $@
--- a/tests/signatures.yaml
+++ b/tests/signatures.yaml
@@ -82,6 +82,83 @@ signature:
              data: !!binary AA==
            - type: padding
 ---
+name: edge_98.0.1108.62_win10
+browser:
+    name: edge
+    version: 98.0.1108.62
+    os: win10
+    mode: regular
+signature:
+    tls_client_hello:
+        record_version: 'TLS_VERSION_1_0'
+        handshake_version: 'TLS_VERSION_1_2'
+        session_id_length: 32
+        ciphersuites: [
+            'GREASE',
+            0x1301, 0x1302, 0x1303, 0xc02b, 0xc02f, 0xc02c, 0xc030,
+            0xcca9, 0xcca8, 0xc013, 0xc014, 0x009c, 0x009d, 0x002f,
+            0x0035
+        ]
+        comp_methods: [0x00]
+        extensions:
+            - type: GREASE
+              length: 0
+            - type: server_name
+            - type: extended_master_secret
+              length: 0
+            - type: renegotiation_info
+              length: 1
+            - type: supported_groups
+              length: 10
+              supported_groups: [
+                  'GREASE',
+                  0x001d, 0x0017, 0x0018
+              ]
+            - type: ec_point_formats
+              length: 2
+              ec_point_formats: [0]
+            - type: session_ticket
+              length: 0
+            - type: application_layer_protocol_negotiation
+              length: 14
+              alpn_list: ['h2', 'http/1.1']
+            - type: status_request
+              length: 5
+              status_request_type: 0x01
+            - type: signature_algorithms
+              length: 18
+              sig_hash_algs: [
+                  0x0403, 0x0804, 0x0401, 0x0503,
+                  0x0805, 0x0501, 0x0806, 0x0601
+              ]
+            - type: signed_certificate_timestamp
+              length: 0
+            - type: keyshare
+              length: 43
+              key_shares:
+                  - group: GREASE
+                    length: 1
+                  - group: 29
+                    length: 32
+            - type: psk_key_exchange_modes
+              length: 2
+              psk_ke_mode: 1
+            - type: supported_versions
+              length: 7
+              supported_versions: [
+                  'GREASE', 'TLS_VERSION_1_3', 'TLS_VERSION_1_2'
+              ]
+            - type: compress_certificate
+              length: 3
+              algorithms: [0x02]
+            - type: application_settings
+              length: 5
+              alps_alpn_list: ['h2']
+            - type: GREASE
+              length: 1
+              data: !!binary AA==
+            - type: padding
+---
 name: firefox_91.6.0esr_win10
 browser:
    name: firefox
--- a/tests/test_impersonate.py
+++ b/tests/test_impersonate.py
@@ -181,6 +181,7 @@ class TestImpersonation:
        [
            # Test wrapper scripts
            ("chrome/curl_chrome98", None, "chrome_98.0.4758.102_win10"),
+            ("chrome/curl_edge98", None, "edge_98.0.1108.62_win10"),
            ("firefox/curl_ff91esr", None, "firefox_91.6.0esr_win10"),
            ("firefox/curl_ff95", None, "firefox_95.0.2_win10"),