502 Bad Gateway: what it means and how to fix it

• The app behind the web server is down or crashed. nginx or another reverse proxy can't get a response from the app server, so it answers 502.
• The connection was refused or reset between the gateway and the server behind it, for example during a deploy or when a process runs out of memory.
• A firewall between them drops the gateway's connections.
• Headers too large for the gateway's buffers, which nginx, for example, rejects with a 502.
• DNS pointing at the wrong place, so the gateway connects to a server that doesn't serve the site.
• On the proxy path: the proxy couldn't connect to the site, or the exit connection dropped in the middle of the request.

There can be two gateways in the path: your proxy, and the site's own CDN or reverse proxy. On an https:// URL, when the 502 arrived tells them apart:

• In reply to CONNECT, before the tunnel opened: your proxy sent it, usually because it couldn't reach the site or the exit failed. curl shows it right after the CONNECT line in verbose output, and Python requests raises a ProxyError.
• Inside the tunnel, as a normal response: the site's side sent it. The proxy can't write a status inside an encrypted tunnel. A Server header naming a CDN or web server usually tells you which layer answered.

For a plain http:// URL, both arrive as normal responses, so the headers and body are your guide. Sending the request without the proxy settles it: if the site answers 502 directly too, the problem is the site.

As a visitor or client

1. Retry after a short wait, with backoff. Many 502s last only while a server restarts.
2. Check whether the proxy or the site sent it (above).
3. If it was the proxy, test the same line against a simple endpoint such as https://api.ipify.org. If that works, the proxy is fine and the site is the problem. If it fails too, check the line's host, port and settings, and that the product has bandwidth left.
4. If the site fails without the proxy as well, it is down. Wait, or let the site owner know.

As the site owner

1. Check that the app server is running and listening where the web server expects it.
2. Read the reverse proxy's error log: nginx, for example, logs the upstream error (connection refused, reset, invalid header) next to each 502.
3. Check that firewalls and security groups let the gateway reach the origin.
4. Look for very large response headers, such as many or large cookies.

# -v prints the CONNECT exchange with the proxy, then the site's response.
curl -v -x "http://USERNAME:PASSWORD@HOST:PORT" https://example.com/ 2>&1 | grep -E "^(> CONNECT|< HTTP)"
# The first "< HTTP" line after "> CONNECT" is the proxy's answer. If it is 200, the tunnel
# opened, and any status after it came from the site's side.

# --retry repeats on timeouts and on 408, 429, 500, 502, 503 and 504 responses.
# curl 7.66.0 and later wait for Retry-After when the server sends one;
# otherwise the wait starts at 1 second and doubles after each try.
curl --retry 5 --retry-max-time 300 -x "http://USERNAME:PASSWORD@HOST:PORT" https://example.com/

import random
import time
from datetime import datetime, timezone
from email.utils import parsedate_to_datetime

import requests

RETRY_STATUSES = {429, 502, 503, 504}


def retry_after_seconds(value):
    """Seconds to wait from a Retry-After header: a number of seconds or an HTTP date."""
    if not value:
        return None
    value = value.strip()
    if value.isdigit():
        return int(value)
    try:
        when = parsedate_to_datetime(value)
        return max(0.0, (when - datetime.now(timezone.utc)).total_seconds())
    except (TypeError, ValueError):
        return None


def get_with_backoff(url, proxies=None, attempts=5, max_wait=120):
    response = None
    for attempt in range(attempts):
        response = requests.get(url, proxies=proxies, timeout=30)
        if response.status_code not in RETRY_STATUSES or attempt == attempts - 1:
            break
        wait = retry_after_seconds(response.headers.get("Retry-After"))
        if wait is None:
            wait = min(60, 2 ** attempt) + random.uniform(0, 1)  # exponential backoff with jitter
        if wait > max_wait:
            break  # a longer pause than this job allows: stop and come back later
        time.sleep(wait)
    return response


proxy = "http://USERNAME:PASSWORD@HOST:PORT"
r = get_with_backoff("https://example.com/", proxies={"http": proxy, "https": proxy})
print(r.status_code)

The function retries only 429, 502, 503 and 504, and raises anything else to you. A requests.exceptions.ProxyError (a refused CONNECT, or a proxy that can't be reached) is raised too, not retried: if it says 407, retrying the same credentials can't succeed.

Residential exits are real consumer connections, so an exit can drop a connection now and then. With a rotating line, a retry leaves from a different IP and takes a different path.

With a sticky session, retries try to stay on the same IP, so if that exit is the problem they can keep failing. Generating a new line starts a new session (sticky sessions). On Core Residential, sticky is on or off and usually keeps the same IP for around 10 minutes; on Premium Residential you set the length, from 3 minutes to 24 hours.

If every exit gets a 502 from one site while other sites work through the same line, the problem is on the site's side, and retrying harder won't fix it.