Minimize the size of the resulting Docker image size by using
multi-stage build and copying the resulting binaries into a minimal
Debian system. This was done with the Alpine Docker images up until now
but not with the Debian images.
Let the user disable the built-in list of HTTP headers when using
libcurl-impersonate, either directly or when replacing it at runtime
with LD_PRELOAD. This is intended to give the user more precise control
over the content and order of the HTTP headers.
To support this, the curl_easy_impersonate() now has an added argument
that can be set to 0, in which case the built-in list of HTTP headers
used by libcurl-impersonate will not be automatically sent. Instead,
the user is expected to supply all the headers by themselves using the
standard CURLOPT_HTTPHEADER libcurl option.
When using LD_PRELOAD to inject libcurl-impersonate, one can disable
the built-in headers by setting the CURL_IMPERSONATE_HEADERS
environment variable to "no".
Cherry pick commit b9b6148c45a00d675d5bb261bf4cbb45468ad807 from
upstream curl to fix a bug in curl 7.84.0 which causes failures with the
'TE: Trailers' header.
Add support for cross-compiling curl-impersonate.
Cross compiling can now be done using the '--host' flag to the configure
script. This will make sure that all sub-components are cross-compiled.
In addition, compiling for a different system requires explicitly
specifying multiple paths used by curl (e.g. for certificates). These
options were added to the configure script as well.
The build and test CI workflow will now attempt to cross-compile
curl-impersonate to ARM64 (aarch64), and upload this binary to the
GitHub release page.
Add the 'configure' script generated by autoconf and its dependencies to
the repository to save the users from having to run 'autoconf' manually.
For easier programmatic access from scripts, etc., store the list of
supported browsers in a browsers.json file. For each browser we store
its details (name, version), which curl-impersonate binary is needed
(chrome vs. firefox) and the name of the wrapper script.
Upload a tar file with libcurl-impersonate compiled for Ubuntu and macOS
automatically for each GitHub release. Previously only a statically
compiled version of the binary curl-impersonate was uploaded.
Build on macOS is failing due to unknown compiler options previously
added to support gcc 12 on Fedora. Add "-Wno-unknown-warning-option" to
silence this failure.
HTTP/2 includes various settings pertaining to stream priorities. Chrome
and Firefox handle them differently, and this behavior was not mimicked
in curl-impersonate well. With this commit, the stream settings set
by curl-impersonate are identical to the real browsers.
* With Chrome, the default stream weight is 256 and the "exclusive bit"
is set to ON.
* With Firefox, a complex tree of stream dependencies is created
by default using PRIORITY frames. This behavior is now mimicked by
curl-impersonate.
* Add compiler flags to the boringssl build step to suppress gcc errors
raised when compiling with gcc 12+, which is present in Fedora. It is not
clear whether this is a compiler bug or a boringssl bug, however the
errors only appear when building in release mode and with gcc 12, so I
don't see any problem with manually supressing them.
* Add documentation for building on Fedora.
Tweak the curl-impersonate build system to make it work on Red Hat
based systems such as CentOS, Fedora and Amazon Linux.
* Change the Makefile to be more portable with different bash versions.
* Detect whether cmake is 'cmake' or 'cmake3', and whether ninja is
'ninja' or 'ninja-build'.
* Explicitly tell brotli to put its libraries in 'lib' dir, otherwise it
might put them in 'lib64' where curl doesn't find them.
* Add instructions for building curl-impersonate from source on Red Hat
based systems.
curl_easy_reset() may be used by an application to reset the options on
a curl handle. If an app has the CURL_IMPERSONATE env var defined, then
the impersonation options are automatically set in curl_easy_init() but
will be cleared in a call to curl_easy_reset(). The desired behavior is
for the impersonation options to be retained (as they are "transparent"
to the user), which this commit takes care of.
Note that this only has an effect when libcurl-impersonate is loaded and
the CURL_IMPERSONATE env var is set. Otherwise the regular behavior of
resetting all the handle options is retained.
Test that the unique TLS signature of curl-impersonate is preserved
after a call to curl_easy_reset() when libcurl-impersonate is loaded.
For this purpose change the 'minicurl' testing util to support multiple
URLs and launch it with 2 different URLs when testing the TLS signature.
.. and for Edge 101 as well. The TLS fingerprint is identical to
previous versions. The HTTP headers have the usual differences in the
user agents. One important change though is in the way the HTTP2
SETTINGS frame is formed. Up until Chrome 98, there was an additional
randomly-generated setting in the frame. This seems to have been removed
since. Therefore it was removed from curl-impersonate as well, and
support for Chrome/Edge 98 was deprecated, since supporting both
signatures requires a lot of work.
