API Performance Comparator

Server assisted Performance

Runs the same bounded sample against two public URLs and presents a side-by-side comparison of latency percentiles, size and status consistency.

Privacy: This tool sends the request through the Mutqan backend relay to reach the target. Only the target URL, headers and body you enter are transmitted, and nothing is stored.

How to use API Performance Comparator

  1. Enter Endpoint A and Endpoint B (for example v1 and v2, or two regions) and optional labels.
  2. Choose requests per endpoint (≤ 25), concurrency and method; add shared headers if needed.
  3. Click "Compare" — the relay measures A, then B, with identical settings.
  4. Read the verdict: which endpoint is faster at the median and whether the difference is significant.
  5. Repeat with more samples if the result is "within noise".

API Performance Comparator features

  • Runs the same bounded sample against two public endpoints, one after the other
  • Side-by-side table of median, average, p90/p95/p99, min/max and throughput
  • Percentage delta and a "better" column per metric
  • Significance hint: is the difference larger than the run's variance?
  • Error rate and status breakdown for both endpoints
  • Chart comparing p50 and p95

API Performance Comparator example

v1 vs v2 of a search endpoint

Input:

A: https://api.example.com/v1/search?q=shoes
B: https://api.example.com/v2/search?q=shoes
Requests: 10 each

Output:

Faster (median): v2 (1.6× at p50)
p50 A → B: -38% · p95 A → B: -41%
Significance: likely real
Median: 412 ms vs 256 ms · p95: 640 ms vs 377 ms

Frequently asked questions about API Performance Comparator

How is the comparison kept fair?

Both endpoints receive the same method, headers, request count and concurrency from the same relay, run one after the other so they do not compete for bandwidth.

What does "within noise" mean?

The median difference is smaller than the run's standard deviation, so it could be random variation. Increase the sample size or repeat the run before drawing conclusions.

Can I compare staging against production?

Yes, as long as both URLs are public and you are allowed to send a handful of requests to them. Keep request counts low for production.

Why compare p95 as well as the median?

Two endpoints can have the same median while one has a much fatter tail. The p95 column shows the experience of the slowest users.

Are response sizes compared?

When the relay reports body sizes they appear as an extra row; otherwise use the Response Size Analyzer on each URL.

Technical notes

Sequential execution avoids the two runs competing for the same relay bandwidth, which would penalise whichever endpoint happened to run second under load. The trade-off is that network conditions can drift between runs; the significance hint compares the median difference with the larger standard deviation to flag results that could be noise.