How do I enable keep-alive connections between Nginx and upstream backends?

Three settings are required: (1) Set proxy_http_version 1.1 in the location block — HTTP/1.0 does not support keep-alive. (2) Set proxy_set_header Connection '' to clear the hop-by-hop Connection header. (3) Add keepalive 64 (or your desired pool size) in the upstream block. This tells Nginx to maintain a pool of idle connections to each upstream, avoiding TCP handshake overhead on every request. Monitor the pool with upstream_keepalive_connections metric.

What is proxy_cache_use_stale and why is it a critical resilience feature?

proxy_cache_use_stale tells Nginx to serve expired cached content when the backend is unhealthy instead of forwarding the error to the client. Configure it with: proxy_cache_use_stale error timeout http_500 http_502 http_503. When the backend returns a 502 or 503, Nginx checks if a stale (expired) cached version exists and serves it with an X-Cache-Status: STALE header. This keeps your API responsive during backend outages for cacheable endpoints, buying time to fix the issue.

What gzip_comp_level should I use for API responses?

Use level 4 for API responses. Levels 1-4 achieve nearly the same compression ratio (typically 70-80% reduction for JSON) but level 4 uses about 40% less CPU than level 9. The diminishing returns above level 4 are not worth the CPU cost for dynamic API responses. Also set gzip_min_length 256 to skip compressing tiny responses where compression overhead exceeds the savings, and ensure gzip_types includes application/json.

How does proxy_cache_lock prevent the thundering herd problem?

When a cached entry expires and multiple simultaneous requests arrive for the same resource, without proxy_cache_lock all of them hit the backend simultaneously — a thundering herd. With proxy_cache_lock on, only the first request fetches from the backend while subsequent requests wait for the cache to be repopulated, then receive the cached response. This dramatically reduces backend load during cache expiration events. Set proxy_cache_lock_timeout to limit how long waiting requests will block before bypassing the lock.

When should I choose Envoy over Nginx for API proxying?

Choose Envoy when building microservice architectures that need: dynamic configuration without restarts (xDS API for service discovery), native gRPC and HTTP/2 proxying, distributed tracing integration (Zipkin, Jaeger), service mesh compatibility (Istio/Linkerd sidecar), and rich per-route observability metrics. Choose Nginx when you need built-in caching (proxy_cache), static file serving, simpler configuration for traditional architectures, and when dynamic reconfiguration is not required. Nginx excels as a monolithic edge proxy; Envoy excels as a distributed sidecar proxy.

Optimizing Reverse Proxies for API Performance