Load Testing

Why it matters

Many performance defects appear only under concurrent load and realistic data volumes: a connection pool that runs out, a lock a hundred checkouts wait for, a query that is fast on ten rows and slow on a million. Functional tests send one request at a time, usually against small test data, and never meet them. If the first real peak finds them, the cost is lost sales, failed bookings or an outage.

For the tester, a load test turns "the system should be fast" into evidence: a defined load, measured results and a verdict against criteria agreed in advance. It supports the release decision, catches performance regressions and gives developers a reproducible run instead of a vague complaint.

Expected load: the number you test against

Expected load is the traffic the system must carry, described with the workload model from the lesson on concurrent users and workload models: which transactions, in what mix, at what rate. Fix it before writing a script, and record where every number came from.

• Production data: the busiest hour of recent months, per transaction, from analytics, access logs or monitoring, not the daily total spread over 24 hours.
• Forecasts: campaigns, seasonal peaks, salary day, registration windows. Multiply the measured peak by the agreed growth factor.
• Levels: test at least the typical busy hour and the expected peak, both loads the system is designed to carry.
• No history yet: estimate from the number of users and how often they act, and label the estimate an assumption in the plan.

Try it: Performance Test Data Generator

The load profile: ramp-up, steady state, ramp-down

The load profile describes how the applied load changes over time. Of its three stages, only one is the measurement. For the store: ramp up to 120 requests per second over 10 minutes, hold for 30, ramp down over 5.

• Ramp-up: load rises gradually, as real traffic does, while caches, connection pools and autoscaling warm up. Starting at full load creates a spike nobody asked about and measures a cold system.
• Steady state: the target load is held constant. The system is steady when throughput matches the applied load and response times and resource use stay flat; response times that keep climbing at constant load mean work is queueing, which is a finding. Hold it long enough for many samples of every transaction, usually tens of minutes; runs of many hours are soak tests.
• Ramp-down: load returns to zero, and response times, queues and resource use should return to baseline. A system that stays slow afterwards has not recovered.

Try it: Concurrent Request Test Generator

Pass criteria: agreed before the run

Pass criteria give a load test its expected result. They come from non-functional requirements, service level objectives or acceptance criteria, cover the performance efficiency characteristic of ISO/IEC 25010 (time behaviour, resource utilisation, capacity) and are agreed with the product owner and developers before the first run, not after seeing the numbers.

• Response time per transaction, as percentiles: search p95 ≤ 300 ms, checkout p95 ≤ 800 ms.
• Error rate: HTTP 5xx and unexpected 4xx responses, timeouts, connection errors and failed content checks below, say, 0.5%.
• Throughput achieved: at least 95% of the planned 120 requests per second. A run that never reached the target proves nothing about it.
• Resource ceilings that keep headroom, such as database CPU below 75%.
• Validity of the test itself: the load generator was not overloaded (CPU below 80%) and responses were checked for content, not only for status 200.
• Window: judge the steady state only. Load tools often compute their built-in thresholds over every sample, ramps included, so filter the steady-state window first.

Planning and running a load test

Write a short plan and review it with the people who own the system and the environment, then run in small steps, so that you trust the script, data and measurements before the full run.

• Plan: scope, transactions and mix, load levels, profile, pass criteria and abort criteria, such as "stop if errors exceed 5% for two minutes".
• Environment and data: production-like size and configuration, not shared during the run, monitored; enough distinct users and products for caches and unique constraints to behave realistically; payment or SMS providers stubbed or in their sandbox.
• Permission: load only systems you own or have written permission to test, from separate generator machines, and tell operations before you start.
• Smoke run, then full run: one or two virtual users for a few minutes prove that the script logs in, uses the data and gets correct responses; then a low-load baseline and the full profile while you watch the dashboards.
• Evaluate and repeat: judge the steady state, repeat the run to confirm the result is reproducible, and record build, configuration, data and time window.

Try it: Load Test Data Generator Try it: Response Time Analyzer Try it: HTTP Request Benchmark

Worked example: an online store before a sale

Release 4.2 is tested before a seasonal sale. Expected peak: 100 requests per second last year × 1.2 growth = 120. The plan and the results of run 1 are in the block below; the tester reads them in this order:

• Validity first: 119.6 requests per second achieved and the generator at 41% CPU, so the timings can be trusted.
• Criteria: search and errors pass; checkout fails, with p95 at 1,140 ms against 800 ms.
• Trend: checkout p95 started near 600 ms and climbed every minute at constant load, so checkout requests were queueing.
• Report and confirm: a defect report with load level, window, metric, graphs and build. Developers trace it to the checkout service's database connection pool, capped at 20 connections; after the fix, the same profile and data run again (confirmation testing) and are compared with run 1.

Load test plan: online store, release 4.2, perf environment
Expected peak:  100 req/s (busiest hour last year) x 1.2 growth = 120 req/s
Mix:            search 70 %, product page 20 %, checkout 10 %
Profile:        ramp-up 0 -> 120 req/s in 10 min, steady state 30 min
                (minutes 10-40), ramp-down 5 min
Pass criteria:  search p95 <= 300 ms, checkout p95 <= 800 ms,
                error rate < 0.5 %, throughput >= 114 req/s
Validity:       load generator CPU < 80 %, response content checked

Run 1, steady state (min 10-40)   Target      Result      Verdict
  throughput                      >= 114      119.6       pass
  search p95                      <= 300 ms   210 ms      pass
  checkout p95                    <= 800 ms   1,140 ms    fail
  error rate                      < 0.5 %     0.2 %       pass
  load generator CPU (max)        < 80 %      41 %        valid

Frequently asked questions

References

• Certified Tester Performance Testing (CT-PT)ISTQB · 2018ISTQB's specialist syllabus on performance testing: basic concepts, performance measurements, test activities and tasks, and tools.
• ISTQB GlossaryISTQBThe standard definitions of testing terms — test case, regression testing, severity, priority, boundary value analysis and the rest.
• Load test types — Grafana k6 documentationGrafana LabsDefinitions of the common load test types — smoke, average-load, stress, soak, spike and breakpoint — and what each one is meant to reveal.
• ISO/IEC 25010:2023 — Systems and software Quality Requirements and Evaluation (SQuaRE) — Product quality modelISO/IEC · 2023The product quality model: functional suitability, performance efficiency, compatibility, interaction capability, reliability, security, maintainability and flexibility.