Add support for impersonating Firefox ESR 91

As suggested on the Hacker News thread
(https://news.ycombinator.com/item?id=30378562), add support for Firefox
Extended Support Release.

The required changes were adding one more cipher to the
ciphers list and changing the user agent. Apart from that the TLS
fingerprint is identical to Firefox 95 which was already supported.
This commit is contained in:
lwthiker
2022-02-18 07:59:53 +02:00
parent 4fe2fd36af
commit b00ad551b6
4 changed files with 55 additions and 12 deletions

View File

@@ -73,6 +73,6 @@ RUN mkdir out && \
cp ${CURL_VERSION}/src/curl out/curl-impersonate
# Wrapper script
COPY curl_ff95 out/
COPY curl_* out/
RUN chmod +x out/*

View File

@@ -27,23 +27,24 @@ docker build -t curl-impersonate .
The resulting image contains:
* `/build/out/curl-impersonate` - The curl binary that can impersonate Firefox. It is compiled statically against libcurl, nss, and libnghttp2 so that it won't conflict with any existing libraries on your system. You can use it from the container or copy it out. Tested to work on Ubuntu 20.04.
* `/build/out/curl_ff95` - A wrapper script that launches `curl-impersonate` with the needed headers and ciphers to impersonate Firefox 95.
* `/build/out/curl_ff91esr` - A wrapper script that launches `curl-impersonate` with the needed headers and ciphers to impersonate Firefox 91 ESR (Extended Support Release).
* `/build/out/curl_ff95` - Same but with Firefox 95.
Copy them from the docker image using `docker cp` or use them in a multi-stage docker build.
In addition install libnss3: `sudo apt install libnss3`. Even though nss is statically compiled into `curl-impersonate`, it is still necessary to install libnss3 because curl dynamically loads `libnssckbi.so`, a file containing Mozilla's list of trusted root certificates. Alternatively, use `curl -k` to disable certificate verification.
## Usage
It is recommended to use the wrapper script `curl_ff95` that adds all the correct headers and flags. For example:
It is recommended to use the wrapper script `curl_ff91esr` that adds all the correct headers and flags. For example:
```
curl_ff95 https://www.google.com
curl_ff91esr https://www.google.com
```
You can add command line flags and they will be passed on to curl. However, some flags change curl's TLS signature which may cause it to be detected.
## Contents
This repository contains the following files:
* [Dockerfile](Dockerfile) - Used to build `curl-impersonate` with all dependencies.
* [curl_ff95](curl_ff95) - Wrapper script that launches `curl-impersonate` with the correct flags.
* [curl_ff91esr](curl_ff91esr), [curl_ff95](curl_ff95) - Wrapper scripts that launch `curl-impersonate` with the correct flags.
* [curl-lib-nss.patch](curl-lib-nss.patch) - The main patch that makes curl use the same TLS extensions as Firefox.
* [libnghttp2-pc.patch](libnghttp2-pc.patch) - Patch to make libnghttp2 compile statically.
* [curl-configure.patch](curl-configure.patch) - Patch to make curl compile with a static libnghttp2.

View File

@@ -1,6 +1,10 @@
--- curl-7.81.0-original/lib/vtls/nss.c 2022-01-03 18:36:46.000000000 +0200
+++ curl-7.81.0/lib/vtls/nss.c 2022-02-17 10:33:28.567798277 +0200
@@ -380,2 +380,91 @@
+++ curl-7.81.0/lib/vtls/nss.c 2022-02-18 07:47:17.612205091 +0200
@@ -145,2 +145,3 @@
{"dhe_dss_des_sha", SSL_DHE_DSS_WITH_DES_CBC_SHA},
+ {"rsa_3des_ede_cbc_sha", TLS_RSA_WITH_3DES_EDE_CBC_SHA},
/* TLS 1.0: Exportable 56-bit Cipher Suites. */
@@ -380,2 +381,91 @@
+/* See nsSSLIOLayerSetOptions@nsNSSIOLayer.cpp, Firefox source code */
+const SSLNamedGroup named_groups[] = {
@@ -92,7 +96,7 @@
+}
+
/*
@@ -1322,2 +1411,20 @@
@@ -1322,2 +1412,20 @@
SECMOD_DestroyModule(module);
+
+ /* Patch for Ubuntu - add a "nss/" suffix to the library name */
@@ -113,7 +117,7 @@
+ SECMOD_DestroyModule(module);
+
return CURLE_FAILED_INIT;
@@ -1923,2 +2030,8 @@
@@ -1923,2 +2031,8 @@
+ if(SSL_SET_OPTION(primary.sessionid)) {
+ if(SSL_OptionSet(model, SSL_ENABLE_SESSION_TICKETS,
@@ -122,7 +126,7 @@
+ }
+
/* enable/disable the requested SSL version(s) */
@@ -1962,2 +2075,10 @@
@@ -1962,2 +2076,10 @@
+ if (set_named_groups(model) != SECSuccess ||
+ set_additional_key_shares(model) != SECSuccess ||
@@ -133,14 +137,14 @@
+ }
+
if(!SSL_CONN_CONFIG(verifypeer) && SSL_CONN_CONFIG(verifyhost))
@@ -2115,2 +2236,6 @@
@@ -2115,2 +2237,6 @@
+ protocols[cur++] = ALPN_HTTP_1_1_LENGTH;
+ memcpy(&protocols[cur], ALPN_HTTP_1_1, ALPN_HTTP_1_1_LENGTH);
+ cur += ALPN_HTTP_1_1_LENGTH;
+
#ifdef USE_HTTP2
@@ -2126,5 +2251,2 @@
@@ -2126,5 +2252,2 @@
#endif
- protocols[cur++] = ALPN_HTTP_1_1_LENGTH;
- memcpy(&protocols[cur], ALPN_HTTP_1_1, ALPN_HTTP_1_1_LENGTH);

38
curl_ff91esr Executable file
View File

@@ -0,0 +1,38 @@
#!/bin/bash
# Find the directory of this script
dir=`echo "$0" | sed 's%/[^/]*$%%'`
PIPE=/tmp/curl-pipe
rm -f "$PIPE" && mkfifo "$PIPE"
exec 5<>"$PIPE" 3>"$PIPE" 4<"$PIPE" 5>&-
# The list of ciphers can be obtained by looking at the Client Hello message in
# Wireshark, then converting it using the cipherlist array at
# https://github.com/curl/curl/blob/master/lib/vtls/nss.c
"$dir/curl-impersonate" \
--ciphers aes_128_gcm_sha_256,chacha20_poly1305_sha_256,aes_256_gcm_sha_384,ecdhe_ecdsa_aes_128_gcm_sha_256,ecdhe_rsa_aes_128_gcm_sha_256,ecdhe_ecdsa_chacha20_poly1305_sha_256,ecdhe_rsa_chacha20_poly1305_sha_256,ecdhe_ecdsa_aes_256_gcm_sha_384,ecdhe_rsa_aes_256_gcm_sha_384,ecdhe_ecdsa_aes_256_sha,ecdhe_ecdsa_aes_128_sha,ecdhe_rsa_aes_128_sha,ecdhe_rsa_aes_256_sha,rsa_aes_128_gcm_sha_256,rsa_aes_256_gcm_sha_384,rsa_aes_128_sha,rsa_aes_256_sha,rsa_3des_ede_cbc_sha \
-H 'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Firefox/91.0' \
-H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8' \
-H 'Accept-Language: en-US,en;q=0.5' \
-H 'Accept-Encoding: gzip, deflate, br' \
-H 'Connection: keep-alive' \
-H 'Upgrade-Insecure-Requests: 1' \
-H 'Sec-Fetch-Dest: document' \
-H 'Sec-Fetch-Mode: navigate' \
-H 'Sec-Fetch-Site: none' \
-H 'Sec-Fetch-User: ?1' \
--http2 --false-start \
$@ >&3
exec 3>&-
IFS= read -d '' -r -n 2 -u 4 header
# Due to the "Accept-Encoding: gzip" header, we may receive a gzipped file.
if [ "$(echo -n $header | xxd -l 2 -p)" == "1f8b" ]; then
(printf "%s" "$header"; cat <&4) | gzip -cd;
else
printf "%s" "$header"; cat <&4;
fi