What a Proxy Really Does

A price-tracking script in Chicago asks a retailer for a product page. Sent directly, the request arrives from a home broadband address in Illinois. Sent through a proxy in Frankfurt, the same request arrives from a German address, and nothing else about it has changed.

The request path, with and without a proxy

Every request carries a return address, because the website has to send the page back somewhere. On a direct connection, that address is the public IP of the network the device sits on. A laptop at home has a private address that only works inside the house. The router swaps it for the one public IP the internet provider assigned.

A proxy is a server that makes the request on the client's behalf. The client hands the request to the proxy. The proxy opens its own connection to the website, fetches the response and passes it back. The return address the website records is now the proxy's.

One connection has become two. On the first, the proxy sees the client's real IP. On the second, the website sees only the proxy. That split is the entire product. The rest of this book is about which IP the proxy lends, and for how long.

A proxy changes the return address on a request, and nothing else.

What the website sees, and what stays the same

The retailer can look up 198.51.100.24 in an IP database before it builds the page. Three facts come back, and each one feeds a decision about whether to serve, challenge or block.

• The IP and its history. Blocklists and reputation scores follow an address around. An IP that sent abuse last week is still paying for it.
• The ASN. This is the network that announces the IP: a cloud host, a home internet provider or a mobile carrier. Chapter 2 explains why this label matters.
• The location. The database returns a country, a region and usually a city. Prices, search results and available content follow from it.

Any browser can show the same three facts for its own connection on ProxyLook's What Is My IP page. Loading it once directly and once through a proxy makes the difference obvious.

The harder lesson is what a proxy leaves alone. First-time buyers often expect a new IP to make them a new visitor. It does not.

A cookie set yesterday identifies the same visitor today from any IP. An account is the same account wherever it logs in from. A browser reports the same screen size, fonts and time zone through every proxy. A Frankfurt IP paired with a Chicago time zone and US English is a mismatch a site can read in one page load.

The TLS fingerprint catches scripts in particular. When a client opens an HTTPS connection, its first message lists ciphers and extensions in an order typical of that software. That message travels through the proxy untouched. A Python script that claims to be Chrome in its user agent still shakes hands like a Python script.

Forward and reverse proxies

Every proxy in this book is a forward proxy. It stands in front of clients and speaks to websites for them. A reverse proxy does the opposite job. It stands in front of a website's servers and speaks to visitors for them.

Nginx balancing load across ten application servers is a reverse proxy. So is Cloudflare. The visitor never chooses it and usually cannot tell it is there. The distinction is practical: the system that inspects and scores an incoming request is often a reverse proxy. Many requests pass through both kinds, a forward proxy on the way out and a reverse proxy on the way in.

Proxy, VPN or Tor

All three put another machine's IP on the request. They differ in scope, in encryption and in how much control the operator gets over the exit.

A VPN suits a person on hotel Wi-Fi who wants everything encrypted. A proxy suits a script that needs a few hundred different German IPs in one afternoon, each on its own connection. Tor suits someone who must hide from the network itself and accepts slow pages in return. The list of Tor exit relays is public, so many sites challenge or block that traffic on sight.

The missing encryption on a proxy sounds worse than it is. On an HTTPS site the content is already encrypted between client and website, and the proxy relays bytes it cannot read. Chapter 3 covers the details.

Anonymity levels live in the headers

Free proxy lists label every entry transparent, anonymous or elite. The labels describe which HTTP headers the proxy adds. Here is a plain HTTP request as a website receives it from a transparent proxy.

GET /product/123 HTTP/1.1
Host: example.com
User-Agent: python-requests/2.32.3
Accept: */*
Via: 1.1 proxy-01
X-Forwarded-For: 203.0.113.7

The last two lines were added by the proxy. Via announces that a proxy handled the request. X-Forwarded-For hands over the client's real IP, which defeats the purpose for anyone trying to change it.

Two caveats keep these labels in proportion. First, they apply to plain HTTP only. On HTTPS the proxy relays an encrypted tunnel and cannot insert a header, so the website receives none. Second, elite describes headers and nothing more. An elite proxy on a flagged IP range is still on a flagged IP range.

The jobs proxies do

Proxies began as office plumbing. A Squid server cached pages and filtered outbound traffic for a whole company, and many still do. The commercial proxy market grew from a different need: seeing the web as someone else, somewhere else, sees it.

Account management is the seventh job. Social media and marketplace teams give each account its own steady IP, so that ten accounts do not share one address. Running multiple accounts can breach a platform's terms of service, and the operator carries that risk.

Each job leans on a different kind of IP. The ProxyLook directory lists more than 200 providers, and the next chapter explains the four proxy types they sell.