# Proxies in DevOps routines By [DeFi (in)security](https://paragraph.com/@defi-in-security) · 2024-07-22 --- > **Due to** [**the Balkanisation of the Internet**](https://script-ed.org/article/data-localisation-and-the-balkanisation-of-the-internet/)**, modern IT specialists have to use various tools to access all the features available on the global network. One major tool is a proxy. Today, we will discuss proxies using the Linux CLI with DevOps applications as examples. We also raise concerns about the security of the process.** > > _1\. Configure Ubuntu VPS to Use Proxies for All CLI Tools that Support Proxy. > 2\. AWS CLI, Helm and Terraform and Proxies. > 3\. Traffic Security when Using Proxy. > 4\. Trust and Verification of the Proxy Server. > 5\. Using Proxy Chains_ 1\. Configure Ubuntu VPS to Use Proxies for All CLI Tools that Support Proxy. ----------------------------------------------------------------------------- To configure an Ubuntu VPS to use an external HTTPS or SOCKS5 proxy with authentication for all outgoing connections, you'll need to set up the environment variables for system-wide proxy settings. Here are detailed steps for both HTTPS and SOCKS5 proxies: ### 1\. Configuring an HTTPS Proxy #### Step 1: Install `curl` and `wget` Ensure `curl` and `wget` are installed to verify the proxy settings. sudo apt update sudo apt install curl wget #### Step 2: Set Environment Variables Set the environment variables for the HTTP and HTTPS proxies. This includes the proxy server address, port, and authentication credentials. Add the following lines to `/etc/environment`: http_proxy="http://username:password@proxy_server:proxy_port" https_proxy="http://username:password@proxy_server:proxy_port" Replace `username`, `password`, `proxy_server`, and `proxy_port` with your actual proxy credentials and server details. For example: http_proxy="http://user:pass@proxy.example.com:8080" https_proxy="http://user:pass@proxy.example.com:8080" After editing, reload the environment variables: source /etc/environment #### Step 3: Verify the Configuration Verify the proxy configuration using `curl` or `wget`: curl -I http://example.com wget -S http://example.com ### 2\. Configuring a SOCKS5 Proxy #### Step 1: Install `tsocks` or `proxychains` Choose one of the utilities to route your traffic through a SOCKS5 proxy. ##### Using `tsocks` Install `tsocks`: Edit the `tsocks` configuration file located at `/etc/tsocks.conf`: sudo nano /etc/tsocks.conf Add the following configuration, replacing `proxy_server`, `proxy_port`, `username`, and `password` with your actual proxy credentials and server details: server = proxy_server server_port = proxy_port server_type = 5 default_user = username default_pass = password ##### Using `proxychains` Install `proxychains`: Edit the `proxychains` configuration file located at `/etc/proxychains.conf`: sudo nano /etc/proxychains.conf Add the following line at the end of the configuration file, replacing `proxy_server`, `proxy_port`, `username`, and `password` with your actual proxy credentials and server details: socks5 username:password@proxy_server proxy_port #### Step 2: Configure Environment Variables for System-Wide Proxy (Optional) To apply the proxy settings system-wide, you may need to wrap your commands with `tsocks` or `proxychains`: tsocks curl -I http://example.com proxychains curl -I http://example.com To use the proxy for all outgoing connections, create aliases for frequently used commands: Add the following lines to your shell configuration file (e.g., `.bashrc`, `.zshrc`): alias curl='tsocks curl' alias wget='tsocks wget' Or for `proxychains`: alias curl='proxychains curl' alias wget='proxychains wget' After editing, reload your shell configuration: source ~/.bashrc ### Notes: * When using environment variables for HTTPS proxies with authentication, your credentials will be exposed in process lists. For better security, consider using tools like `cntlm` for NTLM proxies or other secure methods for proxy authentication. * The above methods will route traffic through the proxy for all commands run in the terminal. For GUI applications, additional configuration may be needed. By following these steps, you can configure your Ubuntu VPS to use an external HTTPS or SOCKS5 proxy with authentication for all outgoing connections. 2\. AWS CLI, Helm and Terraform and Proxies. -------------------------------------------- Yes, `aws-cli`, `helm`, and `terraform` can be configured to use the proxy settings specified in the environment variables. Here's how you can ensure these tools use the proxy settings: ### 1\. Configuring Environment Variables Set the environment variables for HTTP and HTTPS proxies. You can add these lines to `/etc/environment` or your shell configuration file (`~/.bashrc`, `~/.zshrc`, etc.): http_proxy="http://username:password@proxy_server:proxy_port" https_proxy="http://username:password@proxy_server:proxy_port" no_proxy="127.0.0.1,localhost" Replace `username`, `password`, `proxy_server`, and `proxy_port` with your actual proxy credentials and server details. The `no_proxy` variable can be adjusted based on your needs to bypass the proxy for local addresses. After editing, reload the environment variables: source /etc/environment source ~/.bashrc # or source ~/.zshrc ### 2\. AWS CLI The AWS CLI will automatically use the proxy settings specified in the `http_proxy` and `https_proxy` environment variables. To verify, you can run a simple AWS CLI command: aws s3 ls If configured correctly, the AWS CLI will use the proxy settings for this command. ### 3\. Helm Helm also respects the `http_proxy` and `https_proxy` environment variables. You can verify by running a Helm command: helm repo update If Helm uses the proxy, it should update the repositories without issues. ### 4\. Terraform Terraform will use the `http_proxy` and `https_proxy` environment variables for outgoing HTTP and HTTPS connections. To verify, run a Terraform command such as: terraform init If the initialization process completes successfully, Terraform is using the proxy settings. ### 5\. Verifying Proxy Configuration To ensure all these tools are using the proxy, you can use a network monitoring tool like `tcpdump` or check the proxy server logs if you have access. For example, using `tcpdump` to monitor traffic: sudo tcpdump -i eth0 -nn host proxy_server Replace `eth0` with your actual network interface and `proxy_server` with your proxy server's IP address. This will show you traffic going to and from the proxy server. ### Additional Notes 1. **Security**: Be cautious with storing plaintext proxy credentials in environment variables, especially on shared systems. Consider using more secure methods like encrypted configuration files or credential managers if possible. 2. **Tool-specific configurations**: Some tools might have additional configurations for proxies, but generally, setting the environment variables as described will work for most command-line tools. By following these steps, `aws-cli`, `helm`, and `terraform` should all use the specified proxy settings from the environment variables. 3\. Traffic Security when Using Proxy. -------------------------------------- When using a proxy, the security of your traffic depends on the type of proxy and the configuration: ### HTTPS Proxy with Authentication **HTTPS proxies** use TLS/SSL to secure the connection between your client (e.g., a web browser, command-line tool) and the proxy server. Here’s how the security works: 1. **Connection Security**: The connection between your client and the HTTPS proxy server is encrypted using TLS/SSL. This means that anyone intercepting the traffic between your client and the proxy server cannot easily read the data. 2. **Authentication Security**: If the proxy requires authentication, the credentials (username and password) are sent over this encrypted connection, so they are protected from eavesdropping. 3. **End-to-End Encryption**: After the HTTPS proxy server, the traffic is decrypted and then forwarded to the destination server. If the destination server uses HTTPS, then the traffic will be re-encrypted between the proxy server and the destination server, ensuring end-to-end encryption. If the destination server uses HTTP, the traffic will be unencrypted between the proxy server and the destination server. ### SOCKS5 Proxy with Authentication **SOCKS5 proxies** can operate at a lower level (the transport layer), and they can handle various types of traffic (not just HTTP/HTTPS). Here's how the security works: 1. **Connection Security**: SOCKS5 itself does not encrypt traffic. However, it can support a variety of authentication methods. If used with TLS (SOCKS5 over TLS), the connection can be encrypted, but this requires specific support and configuration on both the client and the proxy server. 2. **Authentication Security**: If the SOCKS5 proxy uses username/password authentication, these credentials can be sent in plain text unless an additional encryption layer (like TLS) is used. 3. **End-to-End Encryption**: For end-to-end encryption, the applications using the SOCKS5 proxy must use their own encryption (e.g., HTTPS). This ensures that traffic remains encrypted from the client to the destination, even if it passes through the SOCKS5 proxy unencrypted. ### Summary * **HTTPS Proxy**: Provides encryption between the client and the proxy server using TLS/SSL. Authentication credentials are secure, and if the final destination also uses HTTPS, end-to-end encryption is maintained. * **SOCKS5 Proxy**: Does not provide inherent encryption. To secure traffic, it must be used in conjunction with an encryption protocol like TLS. Authentication credentials can be sent in plain text unless additional measures are taken. ### Ensuring Secure Traffic 1. **Use HTTPS for Sensitive Data**: Always ensure that sensitive data is sent over HTTPS, regardless of the proxy type. This ensures end-to-end encryption. 2. **Verify Proxy Configuration**: Ensure your proxy is properly configured to support TLS/SSL (for HTTPS proxies) or is used in conjunction with an encrypted protocol (for SOCKS5). 3. **Check Proxy Credentials**: Ensure that your credentials are not exposed in plaintext by checking the proxy and client configurations. 4. **Monitor Traffic**: Use tools like `tcpdump` or `Wireshark` to verify that your traffic is encrypted as expected. By following these guidelines, you can ensure that your traffic is secure when using HTTPS or SOCKS5 proxies. 4\. Trust and Verification of the Proxy Server. ----------------------------------------------- You may raise a valid point regarding the trust and verification of the proxy server itself. When using a proxy, especially one that requires authentication and handles sensitive data, it's crucial to ensure that the connection is secure and trusted. Here are steps to mitigate the risk of man-in-the-middle (MitM) attacks and ensure the proxy's authenticity: ### 1\. HTTPS Proxy Verification When using an HTTPS proxy, you can leverage Public Key Infrastructure (PKI) to ensure the proxy server's identity. Here's how: #### Step 1: Certificate Verification Ensure that the HTTPS proxy server uses a valid SSL/TLS certificate issued by a trusted Certificate Authority (CA). You can verify this certificate on the client side. * When connecting to the HTTPS proxy, your client should verify the server's certificate against trusted CAs. * If the proxy's certificate is invalid, expired, or self-signed, your client should reject the connection. #### Step 2: Configure CA Certificates On your Ubuntu VPS, you can configure the system to trust specific CAs or proxy certificates: * **System-wide CA certificates**: * Place your trusted CA certificates in `/usr/local/share/ca-certificates/` and run `sudo update-ca-certificates`. * **Application-specific CA certificates**: * For tools like `curl`, you can specify the CA bundle using the `--cacert` option. Example for `curl`: curl --cacert /path/to/ca-certificates.crt -x https://username:password@proxy_server:proxy_port https://example.com ### 2\. SOCKS5 Proxy Verification SOCKS5 proxies do not inherently support TLS. However, you can ensure secure connections by combining SOCKS5 with TLS or SSH tunneling. #### Step 1: Use SOCKS5 over TLS If the SOCKS5 proxy supports TLS, you can configure your client to establish a TLS connection to the SOCKS5 proxy. This requires both the client and proxy to support this configuration. #### Step 2: SSH Tunneling Another common approach is to use SSH to create a secure tunnel and use it as a SOCKS5 proxy. 1. **Set up SSH tunnel**: ssh -D 1080 -f -C -q -N user@proxy_server * `-D 1080`: Specifies the local SOCKS5 port. * `-f`: Sends SSH to the background after authentication. * `-C`: Enables compression. * `-q`: Quiet mode. * `-N`: Do not execute remote commands. 2. **Configure applications to use the local SOCKS5 proxy** (e.g., `localhost:1080`). ### 3\. DNS Over HTTPS (DoH) and DNS Over TLS (DoT) Ensure DNS queries are secure by using DNS over HTTPS (DoH) or DNS over TLS (DoT): * **DoH**: DNS queries are sent via HTTPS, securing them against eavesdropping and MitM attacks. * **DoT**: DNS queries are encrypted using TLS. ### 4\. Additional Security Measures #### Step 1: Use a VPN A VPN can encrypt all traffic between your client and the VPN server, ensuring that even if a MitM attack occurs on the proxy, the traffic remains encrypted. #### Step 2: Monitor and Audit Connections Regularly monitor and audit your connections using tools like `tcpdump` or `Wireshark` to detect any anomalies or potential MitM attacks. #### Step 3: Use Trusted Proxies Where possible, use proxies that you trust or those provided by reputable providers. Avoid using public proxies for sensitive data. ### Conclusion To ensure that your traffic to the proxy is secure and mitigate the risk of MitM attacks, you should: * Verify the proxy server's certificate using PKI for HTTPS proxies. * Use secure tunneling methods like SSH for SOCKS5 proxies. * Utilize DNS over HTTPS or DNS over TLS to secure DNS queries. * Consider additional layers of security such as VPNs and regular monitoring. By implementing these measures, you can ensure the authenticity and security of your connections through proxies. 5\. Using Proxy Chains. ----------------------- If you are not able to connect to your proxy directly you have to use chain of proxies. Configuring a chain of proxies directly through environment variables on a Linux system isn't natively supported by most tools. Typically, environment variables like `http_proxy`, `https_proxy`, and `no_proxy` only support single proxy configurations. However, you can achieve proxy chaining using specialized tools or configurations. Here are a few methods: ### 1\. Using `proxychains` `proxychains` is a versatile tool that allows you to chain multiple proxies together. It supports HTTP, SOCKS4, and SOCKS5 proxies. #### Installation sudo apt update sudo apt install proxychains4 #### Configuration Edit the configuration file `/etc/proxychains4.conf`: sudo nano /etc/proxychains4.conf Add your proxy chain at the end of the file. For example: # ProxyList format # type host port [user pass] # (socks4|socks5|http) host port [user pass] [ProxyList] http proxy1.example.com 8080 user1 pass1 socks5 proxy2.example.com 1080 user2 pass2 http proxy3.example.com 3128 user3 pass3 #### Usage Run your command with `proxychains`: proxychains4 curl http://example.com proxychains4 wget http://example.com proxychains4 aws s3 ls proxychains4 helm repo update proxychains4 terraform init ### 2\. Using `tsocks` `tsocks` is another tool that can be used for SOCKS proxy chaining. It does not support HTTP proxies natively but can be combined with other tools. #### Installation #### Configuration Edit the configuration file `/etc/tsocks.conf`: sudo nano /etc/tsocks.conf Add your proxy chain: server = proxy1.example.com server_type = 5 server_port = 1080 default_user = user1 default_pass = pass1 server = proxy2.example.com server_type = 5 server_port = 1080 default_user = user2 default_pass = pass2 #### Usage Run your command with `tsocks`: tsocks curl http://example.com tsocks wget http://example.com tsocks aws s3 ls tsocks helm repo update tsocks terraform init ### 3\. Custom Scripts You can create custom scripts to handle proxy chaining for specific tools. For example, using `curl` with a chain of proxies: #!/bin/bash PROXY1="http://user1:pass1@proxy1.example.com:8080" PROXY2="http://user2:pass2@proxy2.example.com:8080" PROXY3="http://user3:pass3@proxy3.example.com:3128" URL=$1 curl -x $PROXY1 -x $PROXY2 -x $PROXY3 $URL Save this script as `chaincurl.sh`, make it executable, and use it: chmod +755 chaincurl.sh ./chaincurl.sh http://example.com ### 4\. Chaining Proxies in Web Browsers If you need proxy chaining in web browsers, tools like FoxyProxy can be used to configure multiple proxies. ### Summary To chain proxies and use them with Linux CLI tools: * Use `proxychains` for versatile proxy chaining with support for HTTP, SOCKS4, and SOCKS5. * Use `tsocks` for SOCKS proxy chaining. * Create custom scripts for specific tools like `curl`. * Note that chaining proxies via environment variables alone is not directly supported. By using these methods, you can configure and utilize a chain of proxies for your Linux CLI tools effectively. --- *Originally published on [DeFi (in)security](https://paragraph.com/@defi-in-security/proxies-in-devops-routines)*