Which HTTP 5xx status codes should I automatically retry?

Always retry 502 Bad Gateway, 503 Service Unavailable, and 504 Gateway Timeout as these indicate transient infrastructure issues. Retry 500 Internal Server Error only on idempotent operations (GET, PUT, DELETE) since the cause may be a transient dependency failure. Never retry 501 Not Implemented, 505 HTTP Version Not Supported, or 508 Loop Detected, as these reflect permanent server configuration issues.

What is a retry budget and how does it prevent retry storms?

A retry budget caps total retries at a percentage of successful requests within a sliding time window, typically 10% as recommended by Google SRE. For example, if a service handles 1,000 requests per minute, only 100 retries are allowed in that window. This prevents cascading failures where thousands of clients retrying simultaneously overwhelm an already struggling server.

How do idempotency keys make POST request retries safe?

An idempotency key is a unique identifier (typically a UUID) sent with each request in a header like Idempotency-Key. The server stores the key with its response. If the same key appears again because the client retried, the server returns the cached response instead of processing the request a second time. This prevents duplicate side effects like double charges or duplicate record creation.

How should I parse the Retry-After header from a 503 response?

The Retry-After header comes in two formats: an integer representing delay in seconds (e.g., Retry-After: 120) or an HTTP-date for an absolute time (e.g., Retry-After: Wed, 21 Oct 2025 07:28:00 GMT). Try parsing as a number first; if that fails, parse as a date and compute the difference from now. Always prefer the server Retry-After value over your own calculated backoff delay.

When should I use a circuit breaker instead of just retrying?

Use a circuit breaker when failures are sustained rather than transient. If a dependency is down for minutes (not milliseconds), retries waste resources and add latency. A circuit breaker trips open after a threshold of failures (e.g., 5 consecutive), immediately rejects requests for a cooldown period (e.g., 30 seconds), then allows a single probe request. If the probe succeeds, traffic resumes. This protects both the client from pointless waiting and the server from retry-amplified load.

Retry Mechanisms for Temporary Server Errors