When integrating cURL with proxy services, you must first access the proxy. The syntax for the cURL proxy URL is structured as follows:

[<PROTOCOL>://][<USERNAME>:<PASSWORD>]@[:<PORT>]

This includes:

• <Protocol>: The protocol for connecting to the proxy server. If unspecified, cURL defaults to http://.

• <Host>: The IP address or URL required for the proxy server hostname.

• <Port>: The port number the proxy server is listening on. If unspecified, cURL defaults to using port 1080.

• <Username>: An optional username required for authentication.

• <Password>: An optional password required for authentication.

The most commonly used cURL proxy protocols are HTTP and HTTPS, followed by SOCKS.

For instance, if your proxy server's IP address is 192.168.1.1 and operates on port 8080, the command would be:

curl -x http://192.168.1.1:8080 http://example.com

This command directs cURL to route the request for http://example.com through the proxy server 192.168.1.1 on port 8080.

Managing Proxy Authentication

Some proxy servers require a username and password for authentication. cURL supports proxy authentication, allowing web scrapers to access these proxy servers while adhering to their security measures.

To connect to a proxy server with authentication, use the --proxy-user option to provide the username and password:

curl --proxy http://proxy-url.com:8080 --proxy-user user:pass http://target-url.com/api

This command verifies the provided username and password with the proxy server to send the HTTP request to the target URL through the specified proxy.

Additionally, include a proxy-authorized title in the request header using the --proxy-header option:

curl --proxy http://proxy-url.com:8080 --proxy-user user:pass --proxy-header "Proxy-Authorization: Basic dXNlcjEyMzpwYXNzMTIz" http://target-url.com/api

This command instructs cURL to use the specified username and password for authentication with the proxy server.

Using Environment Variables for Proxies

For regular usage of a cURL proxy, setting it as an environment variable can be more convenient. cURL allows the use of environment variables for each supported protocol. For instance, if http_proxy and https_proxy are set, cURL will automatically use them for the appropriate protocol:

export http_proxy=http://192.168.1.1:8080 export https_proxy=http://192.168.1.1:8080

You can also include authentication if necessary:

export http_proxy=http://username:password@192.168.1.1:8080 export https_proxy=http:///username:password@192.168.1.1:8080

……

More can read: