Percentiles: p50, p95 and p99

Why it matters

Users do not experience an average; they remember their slow requests. A few very slow requests can hide behind a healthy-looking average, so "average 220 ms" can describe a release in which one request in ten takes over a second.

Percentiles make performance testable. "The API should be fast" cannot fail; "p95 below 500 ms at 200 requests per second" can. They also let you compare two builds fairly and describe a regression in terms a developer can act on.

What a percentile is and how to calculate it

Sort every response time measured for one operation from fastest to slowest. The pth percentile is the value at or below which p percent of them lie. By hand, use the nearest-rank method: rank = ceil(p / 100 × n), that is p percent of the count rounded up, and read the value at that rank.

With 20 measurements, p50 is the value at rank 10, p95 at rank 19 and p99 at rank 20, which is simply the slowest request. A percentile is only as precise as the data behind it: p99 rests on the slowest 1%, just 2 requests out of 200, so it needs hundreds or thousands of samples.

• Some tools use the nearest rank, others interpolate between neighbouring values, so with few samples two tools can report different numbers for the same run. With thousands of samples the difference is small.
• Labels differ: k6 writes p(95), and the Aggregate Report in JMeter has a 95% Line column. Check which percentiles a summary shows; in k6 you choose them with the summaryTrendStats option.
• Compute percentiles per operation: a 5 ms health check and a 400 ms search in one list give a p95 that describes neither.

Try it: Response Time Analyzer

What p50, p95 and p99 each tell you

Each percentile answers a different question, so reports show several side by side. And "only 1%" is more than it sounds: at 50 requests per second it means 30 slow requests a minute, and if a page makes 10 independent calls, each with a 1% chance of being slower than its p99, the chance that at least one is slow is 1 - 0.99^10, about 10%.

• p50, the median: half the requests were faster, half slower. It shows the typical experience and moves when a change slows everything down, such as a query that lost its index.
• p95: one request in 20 was slower. It is a common choice for test targets and SLOs because it covers almost every request and is less sensitive to a single outlier than p99.
• p99: one request in 100 was slower. This is the tail, where garbage-collection pauses, lock contention, cold caches, retries and slow dependencies show up. It needs many samples and often rises before the median as load grows.
• The maximum is a single request: check it for timeouts, but it is too noisy to be a target. High-traffic services also track p99.9.

Why averages mislead

Response times are skewed: most requests cluster near the fast end, and a long tail of slow ones stretches towards high values. The mean adds every value and divides by the count, so a few slow requests pull it up, yet it says nothing about how many users waited.

Two services each handle 100 requests with an average of 200 ms. Service A answers 20 requests each in 180, 190, 200, 210 and 220 ms. Service B answers 88 requests in 100 ms, 7 in 600 ms, 4 in 1,250 ms and one in 2,000 ms. A has p50 200 ms, p95 220 ms and p99 220 ms; B has p50 100 ms, p95 600 ms and p99 1,250 ms. B is faster for most users and much slower for the unlucky ones, and only A meets a p95 target of 500 ms.

• An average can describe a request that never happened: if 80% of requests hit a cache in 20 ms and 20% miss and take 800 ms, the mean is 176 ms, a time no request took.
• You cannot average percentiles. The p95 of two servers, or of a day of one-minute windows, is not the mean of their p95 values; compute it from the raw measurements or a merged histogram (counts of requests per latency range).
• An error returned in 5 ms can pull the average and the percentiles down and make a failing system look fast. Report the latency of successful requests and the error rate side by side.

Try it: API Performance Comparator

Worked example: 20 search requests

After a release, a tester sends 20 sequential requests to the search API on staging, with no other load, to check the team's target: p95 at or below 500 ms. The block below holds the raw times, the sorted list and the calculation.

• Mean: 4,400 / 20 = 220 ms. It looks comfortable, yet 17 of the 20 requests were faster than the average and two were far slower.
• p50: rank 10 → 125 ms. The typical search is quick.
• p95: rank 19 → 600 ms. The target is not met, and the second-slowest request decides it.
• p99: rank 20 → 1,400 ms, just the maximum. Twenty samples cannot support a p99, so do not report one.
• The report gives p50, p95 and the maximum with the sample size and load, states that the target is not met, names the slow requests (the 4th and the 10th) for investigation, and asks for a rerun with several hundred requests under the agreed load before any decision.

Search API, staging, 20 sequential requests, response time in ms
Target: p95 <= 500 ms

As measured:  128 113  96 600 122 136 104 150 117 1400
              110 131 194 101 125 240 142 108 163  120

Sorted:
  rank    1    2    3    4    5    6    7    8    9   10
  ms     96  101  104  108  110  113  117  120  122  125
  rank   11   12   13   14   15   16   17   18   19   20
  ms    128  131  136  142  150  163  194  240  600 1400

Mean  = 4400 / 20                   = 220 ms
p50   = rank ceil(0.50 x 20) = 10  -> 125 ms
p95   = rank ceil(0.95 x 20) = 19  -> 600 ms   (target 500 ms: not met)
p99   = rank ceil(0.99 x 20) = 20  -> 1400 ms  (= the maximum)

Try it: HTTP Request Benchmark

Writing latency targets and SLOs with percentiles

In the Google SRE book, a service level indicator (SLI) is a carefully defined measurement of the service, such as request latency, and a service level objective (SLO) is the target value or range for it. A service level agreement (SLA) is a contract with consequences when the objective is missed. Latency objectives use percentiles because they must hold for nearly all users, not for an average one.

A complete target names five things: which requests (the operation; successful only or all), the percentile, the threshold, the window or test duration, and the conditions such as load. "95% of requests complete within 500 ms" and p95 ≤ 500 ms are the same condition. In k6 a criterion becomes thresholds: when one is missed, the run fails with a non-zero exit code, so a CI pipeline can stop on it. The script at the end of this section turns the load-test criterion below into k6: its scenario starts 200 requests per second whatever the response time, drawing on a pool of 100 to 400 virtual users (preAllocatedVUs, maxVUs) to send them; the next lesson, on concurrent users and workload models, explains this open model and how to size the pool. The {expected_response:true} filter limits the latency thresholds to successful responses.

• Production SLO: "99% of successful GET /products/{id} requests complete within 300 ms, measured over a rolling 28 days."
• Two levels for checkout: "p95 ≤ 800 ms and p99 ≤ 2 s for POST /checkout", bounding both the typical slow request and the tail.
• A load-test criterion derived from an SLO: "For GET /search at 200 requests per second for 30 minutes: p95 ≤ 500 ms and p99 ≤ 1200 ms for successful requests, error rate below 0.5%." The numbers in these examples are illustrations; real ones come from user needs and the product team, not from a standard.

import http from 'k6/http';

export const options = {
  scenarios: {
    search: {
      executor: 'constant-arrival-rate', // 200 iterations a second, one request each
      rate: 200,
      timeUnit: '1s',
      duration: '30m',
      preAllocatedVUs: 100,
      maxVUs: 400,
    },
  },
  thresholds: {
    'http_req_duration{expected_response:true}': ['p(95)<=500', 'p(99)<=1200'],
    http_req_failed: ['rate<0.005'],
  },
};

export default function () {
  http.get('https://staging.example.com/search?q=lamp');
}

Frequently asked questions

References

• Site Reliability Engineering: How Google Runs Production SystemsGoogle / O'Reilly Media · 2016Service level objectives, error budgets, monitoring the "four golden signals" and handling overload — the operational side of resilience.
• Monitoring Distributed Systems — Site Reliability Engineering (Rob Ewaschuk)Google / O'Reilly Media · 2016Google's SRE guidance on monitoring: black-box versus white-box monitoring and the four golden signals — latency, traffic, errors and saturation.
• Grafana k6 documentationGrafana LabsOfficial documentation for Grafana k6, an open-source and extensible load and performance testing tool.
• Apache JMeter User's ManualApache Software FoundationHow to build, parameterise and run load tests with JMeter, and how to read its listeners and reports.