Why are averages misleading for API latency monitoring?

Averages hide the distribution of latency. An API with 100ms average might have 99% of requests at 50ms and 1% at 5000ms. The 1% experiencing 5-second responses are real users with real frustration, but the average looks fine. Percentiles (p50, p95, p99) reveal the actual distribution. The p99 shows what the slowest 1% of users experience, which is critical for identifying tail latency problems caused by cache misses, GC pauses, or slow dependency calls.

Should I use Prometheus histograms or summaries for latency?

Use histograms in almost all cases. Histograms store observation counts in configurable buckets and allow you to compute any percentile at query time. Crucially, histogram buckets can be aggregated across multiple instances using sum by (le), which is essential for distributed services. Summaries compute percentiles client-side and cannot be aggregated across instances. Use summaries only when you need exact percentiles from a single instance. The trade-off is that histogram quantile accuracy depends on bucket boundaries, so choose buckets that cover your expected latency range.

What is an Apdex score and how do I calculate it?

Apdex (Application Performance Index) converts latency into a 0-to-1 satisfaction score using a threshold T. Requests completing within T are 'satisfied' (count as 1), within 4T are 'tolerating' (count as 0.5), and above 4T are 'frustrated' (count as 0). The formula is (satisfied + tolerating/2) / total. With T=200ms: a 150ms request is satisfied, a 600ms request is tolerating, a 1200ms request is frustrated. An Apdex of 0.94+ is excellent, 0.85-0.93 is good, 0.70-0.84 is fair, below 0.70 is poor. It's useful for communicating performance to non-technical stakeholders.

How should I set histogram bucket boundaries for HTTP latency?

Choose buckets that provide resolution where your latency variance lives. For typical web APIs, use fine-grained buckets between 10ms and 1s: [0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10]. Avoid the default Prometheus buckets which are too coarse for API latency. The key principle: histogram_quantile interpolates linearly within each bucket, so narrower buckets give more accurate percentile estimates. If your p99 target is 500ms, ensure you have multiple buckets around that boundary (e.g., 0.25, 0.5, 1.0).

How do I use distributed tracing to find the slow component in a request?

Distributed tracing (OpenTelemetry, Jaeger, Zipkin) breaks each request into spans representing work in each service and component. When a request takes 3 seconds, the trace shows a waterfall view: API gateway 5ms, auth 15ms, order service 50ms, payment API 2800ms. The longest span is your bottleneck. To identify systemic issues (not just one slow request), query your tracing backend for spans with duration above a threshold and group by service and operation. This reveals which dependency consistently contributes the most latency across all requests.

Monitoring API Latency and Timeouts