Ameh
Resume
SYSTEM DESIGN

Reverse Proxies
The Invisible Middleman

Every production system you use — from Netflix to GitHub — hides its servers behind a reverse proxy. This invisible layer absorbs TLS, balances load, caches responses, and shields your infrastructure. Here is how it works, and why it matters.

CONCEPT

What a Reverse Proxy Does

A reverse proxy is a server that sits in front of one or more backend servers, intercepting requests from clients on the internet before they ever reach the application code. The client thinks it is talking directly to your app — it has no idea backend servers exist at all.

Core responsibilities

  • →Load balancing — distribute incoming traffic across a pool of backend servers so no single node is overwhelmed.
  • →SSL/TLS termination — decrypt HTTPS at the proxy so backend servers communicate over fast, unencrypted HTTP internally.
  • →Static content offloading — serve images, JS bundles, and CSS directly from the proxy or a cache, never hitting the app server.
  • →Request buffering — absorb slow client uploads so backends receive complete requests at full speed.
  • →IP anonymisation — backend servers only see the proxy's IP; real client IPs are forwarded in headers like X-Forwarded-For.
  • →Health checks — continuously probe backends and remove unhealthy nodes from the rotation automatically.

Popular reverse proxy software includes Nginx, HAProxy, and Envoy. Cloud providers wrap the same concept into managed products: AWS ALB, GCP Cloud Load Balancing, and Cloudflare Workers all behave as reverse proxies at scale.

ARCHITECTURE

Reverse Proxy vs Forward Proxy

People often confuse the two because both are called proxies. The critical difference is whose interests they serve.

Forward Proxy

Sits on the client side. The client explicitly routes its requests through it. The server never knows the real client IP.

  • → Corporate web filters
  • → VPNs and anonymisers
  • → School / office internet gateways

Reverse Proxy

Sits on the server side. The client has no idea it exists. The backend servers are completely hidden from the internet.

  • → Nginx in front of your Node app
  • → AWS ALB in front of ECS tasks
  • → Cloudflare in front of your origin

Mental model: a forward proxy acts on behalf of the client and hides it from the server. A reverse proxy acts on behalf of the server and hides it from the client. The word "reverse" refers to whose side the proxy is on, not which direction traffic flows.

TRAFFIC

Routing Strategies

The reverse proxy decides which backend receives each request. The right strategy depends on how uniform your workloads are and whether sessions need to be sticky.

Round Robin

Requests cycle through the server pool in order: 1 → 2 → 3 → 1 → 2 → …. Simple, predictable, and correct when every server is the same spec and every request costs roughly the same.

# Nginx round-robin (default)
upstream backend {
    server app1.internal:8080;
    server app2.internal:8080;
    server app3.internal:8080;
}

server {
    location / {
        proxy_pass http://backend;
    }
}

Least Connections

Each new request goes to the server currently handling the fewest active connections. This adapts automatically when requests vary in duration — a slow database query on server 1 means it receives fewer new requests until it recovers.

# Nginx least_conn
upstream backend {
    least_conn;
    server app1.internal:8080;
    server app2.internal:8080;
    server app3.internal:8080;
}

IP Hash (Sticky Sessions)

hash(client_ip) % N maps each client to a specific server consistently. Useful when server-side session state is not shared — the same user always lands on the same node. The tradeoff: one server becomes a hot spot if a heavy user (or IP range) hash to it.

# Nginx ip_hash — sticky sessions
upstream backend {
    ip_hash;
    server app1.internal:8080;
    server app2.internal:8080;
    server app3.internal:8080;
}

# HAProxy equivalent: balance source
# balance source
# hash-type consistent

Weighted Round Robin

Assign a weight to each server proportional to its capacity. A server with weight=3 receives 3× as many requests as one with weight=1. Essential when deploying a mix of instance sizes or during gradual canary rollouts.

# Nginx weighted — large instance gets 3x traffic
upstream backend {
    server app-large.internal:8080  weight=3;
    server app-small.internal:8080  weight=1;
}
PERFORMANCE

SSL Termination & Static Offloading

Two of the highest-value jobs a reverse proxy performs are ones your application code should never have to handle.

🔒 SSL / TLS Termination

Public HTTPS traffic hits the proxy, which holds the TLS certificate and decrypts it. Backend servers receive plain HTTP — no TLS overhead, no certificate management per service. Benefits:

  • →Single certificate renewal point instead of one per microservice.
  • →CPU-intensive cryptography runs on optimised proxy hardware, not your app server.
  • →Internal traffic on a private network can safely remain unencrypted.
  • →Enables HTTP/2 and HTTP/3 multiplexing at the edge without app changes.

⚡ Static Content Offloading

The proxy serves JS, CSS, images, and fonts straight from disk or an in-memory cache. The backend receives only the requests it actually needs to process.

# Nginx — serve static files directly, proxy the rest
server {
    root /var/www/html;

    # Static assets — served from disk, never hit Node
    location ~* .(js|css|png|jpg|svg|woff2)$ {
        expires 1y;
        add_header Cache-Control "public, immutable";
    }

    # Everything else — forwarded to the app
    location / {
        proxy_pass http://backend;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}

Combining SSL termination and static offloading can reduce backend CPU usage by 30–60 % on typical web apps — without changing a line of application code.

EXPERIMENT

Live Traffic Simulation

Adjust the sliders and strategy dropdown to see how the reverse proxy routes packets to backend servers in real time. Kill a server to see the proxy automatically route around the failure.

WARNING

Reverse Proxy Pitfalls

  • →Single point of failure: a proxy without redundancy is as fragile as having no load balancing. Always run at least two proxy instances behind a DNS failover or anycast VIP.
  • →Misconfigured timeouts: if the proxy's read timeout is shorter than your longest legitimate request (e.g. a file export), the proxy silently drops connections mid-flight. Tune timeouts carefully.
  • →X-Forwarded-For spoofing: if your app trusts the X-Forwarded-For header naively, an attacker can inject a fake IP. Only trust the header when it comes from a known proxy; use a trusted IP list.
  • →WebSocket blindspot: HTTP reverse proxies drop WebSocket upgrade handshakes unless explicitly configured. Add 'proxy_http_version 1.1' and the Upgrade/Connection headers.
  • →TLS inspection privacy: terminating TLS at the proxy means the proxy sees plaintext traffic. Ensure your proxy host is in a secure, access-controlled environment with audit logging.
  • →Sticky session hotspots: IP hash and cookie-based stickiness can concentrate load on one server when a large ISP's NAT shares one IP across thousands of clients. Monitor per-server load.
  • →Missed health check tuning: health checks that are too aggressive cause flapping; checks that are too lenient leave unhealthy servers in rotation. Tune intervals and thresholds for your SLA.
SUMMARY

Key Takeaways

  • →A reverse proxy sits between clients and backends, hiding server topology and acting on the server's behalf — unlike a forward proxy which acts on the client's behalf.
  • →SSL termination at the proxy centralises certificate management and offloads TLS CPU cost from application servers.
  • →Round-robin works for uniform workloads; least-connections adapts to variable request durations; IP hash maintains sticky sessions at the cost of potential hotspots.
  • →Static content offloading can halve backend CPU usage without touching application code.
  • →Health checks let the proxy automatically route around failed or slow backends — the foundation of zero-downtime deployments.
  • →Always run proxies in an HA pair; a single proxy instance is a single point of failure that negates the fault-tolerance of a redundant backend pool.
  • →Tune WebSocket headers, timeouts, and X-Forwarded-For handling explicitly — the defaults are often wrong for production traffic.