Module 3 · Testing fundamentals

System Testing

By the Mutqan Soft QA team · · 18 min · Beginner

System testing is the test level that checks a complete, integrated system against its specified requirements. The test object is the whole product as a user or another system meets it: front end, back-end services, database, configuration and background jobs, deployed and running together.

For example, a system test on an event-ticket website signs in as a customer, books two seats, pays in the payment provider’s test mode, and checks that the booking appears under "My tickets" and the e-ticket arrives by e-mail. The same level checks how well the system behaves: speed under load, the Arabic and English versions, and who can see which booking.

Why it matters

Component and integration tests show that the parts work and talk to each other. They cannot show that the assembled product does what its requirements say. Many failures appear only when everything runs together with a realistic configuration: seats held for hours because two servers use different time zones, or a customer who opens someone else’s booking by changing a number in the URL. If system testing does not find them, customers will.

For a tester, this is where an independent view counts most: system tests come from the requirements and risks, not the code, and their results are the evidence for deciding whether a build is ready for acceptance testing.

What system testing checks, and against what

The ISTQB Foundation syllabus (CTFL v4.0) describes system testing as focusing on the overall behaviour and capabilities of an entire system or product, often including functional testing of end-to-end tasks and non-functional testing of quality characteristics, and possibly performed by an independent test team. The interfaces to other systems, such as a payment provider, are the focus of system integration testing from the previous lesson.

The test basis is the system-level specification: functional and non-functional requirements, use cases, user stories, business rules, risk analysis and the user documentation. Tests run through the system’s external interfaces — the user interface, the public API, the files and messages it exchanges — not by calling internal functions. Typical failures at this level:

  • A rule spanning several components gives a wrong result, such as a discount applied twice between cart and invoice.
  • An end-to-end task cannot be completed: the booking is saved, but no confirmation is sent.
  • A failure caused by configuration or deployment rather than code: a missing setting, a wrong time zone, an expired certificate.

Functional and non-functional testing at system level

Functional system tests check what the system does: each requirement, business rule and end-to-end task, including invalid input and error handling, not just the main flow. Non-functional system tests check how well it does it, against quality characteristics such as performance efficiency, security, compatibility, reliability and usability (ISO/IEC 25010:2023 calls usability "interaction capability"). System testing needs both.

Some characteristics can only be judged on the complete system: response time depends on every layer and the network between them, and ease of use on the whole journey. ISTQB therefore notes that some non-functional characteristics are best tested on a complete system in a representative test environment. They also need measurable requirements: "the seat map is fast" cannot pass or fail; "loads within 2 seconds for 95% of requests with 500 concurrent users" can.

A production-like test environment

A system test result is only as trustworthy as the environment it ran in: a test that passes there should pass in production, and a failure there should point to the product, not the set-up. ISTQB speaks of a representative test environment; teams call it staging, pre-production or the system test environment. Representative does not mean identical: keep everything that affects behaviour the same, make unavoidable differences on purpose, and record every difference.

  • Same: the build artefact that will be released, deployed with the same scripts; operating system, runtime and database versions; settings such as feature flags, time zone and scheduled jobs.
  • Different on purpose, data: synthetic data, or, where policy allows, masked copies of production data, in which names, contact details and other personal data are replaced; either way with realistic shape and volume. Never unmasked personal data of real customers.
  • Different on purpose, external services: payment, SMS and e-mail providers in sandbox or test mode, or simulated, so no real money moves and no real customer gets a message. ISTQB notes that simulating sub-systems is possible at this level.
  • Often different, capacity: a smaller environment still finds functional defects, but its performance results do not transfer directly to production.
  • Accounts: a test account for every role, such as customer, organiser and administrator.
Ticket site, release 2.3: the system test environment runs the same build, versions and settings as production, while data, payment and messaging providers and often capacity differ on purpose and are recorded.

Production

Real customers, real payments, real messages

Build:
Release 2.3 as shipped
Versions and settings:
Operating system, runtime, database engine, time zone, feature flags, scheduled jobs
Data:
Real customers, bookings and payments
Payment, e-mail and SMS:
Live providers
Capacity:
The full number of application servers
Accounts:
Real customers, organisers and administrators

System test environmentStaging or pre-production

Representative of production and safe to test in

Build:
The same build artefact, deployed with the same scripts
Versions and settings:
The same; any change is a recorded difference
Data:
Synthetic or masked, with a realistic shape and volumeCaution
Payment, e-mail and SMS:
Sandboxes or simulators: no real money, no real messagesCaution
Capacity:
Half the servers: performance results are reported with the sizeCaution
Accounts:
A test account for every role

Caution marks a deliberate difference from production: acceptable, but written into the test report so that results are read correctly.

Try it: Test Data Factory Try it: Fake User Generator

Worked example: system tests for an event-ticket site

Release 2.3 of a ticket-booking website has five requirements. FR-1: a signed-in customer can book 1 to 6 tickets per event and receives an e-ticket by e-mail. FR-2: seats in an unpaid checkout are held for 10 minutes, then released. NFR-1: the seat map loads within 2 seconds for 95% of requests with 500 concurrent users. NFR-2: customers see only their own bookings. NFR-3: the site works in Arabic and English.

Integration testing has passed, and a smoke test, a short run of the most important functions, shows that the release candidate is testable. The tester designs system tests from the requirements, each traced to one:

  • ST-01 (FR-1, functional): book 2 seats and pay in test mode → booked; e-ticket for 2 seats received.
  • ST-02 (FR-1, functional): try to book 7 tickets → refused with a clear message, nothing reserved.
  • ST-03 (FR-2, functional): open checkout and do not pay → seats free again after 10 minutes.
  • ST-04 (NFR-1, performance): 500 virtual users, simulated by a load-test tool, open the seat map → at least 95% load within 2 seconds.
  • ST-05 (NFR-2, security): customer A opens customer B’s booking link → access refused, no data shown.
  • ST-06 (NFR-3, localisation): book in Arabic in two supported browsers → right-to-left layout and an Arabic e-ticket.

Try it: Requirements → Test Cases Try it: Test Scenario Generator

Reading the results

ST-03 failed: the seats were released after 3 hours and 10 minutes. The web server saved the hold time in Riyadh local time (UTC+3) without a time zone, and the release job, on a server set to UTC, read it as UTC. Every component test had passed; only the deployed whole system showed the failure. The defect report names the build, the environment and both server settings.

ST-04 passed, but the environment has half the production servers, so the report states this next to the result. The system test report then gives results per requirement, open defects with their severity (the degree of impact each one has) and the environment differences: the evidence for deciding whether the build moves on to acceptance testing.

Common mistakes

  • Running only the main flow, like a demo, and skipping invalid input and error handling.
  • Leaving performance, security and compatibility "for later", when the release date leaves no later.
  • Testing in an environment that differs from production without recording how, or blaming every failure on "the environment" without checking.
  • Copying real customers’ personal data into the test environment instead of using synthetic or masked data.
  • Writing system tests that do not name the requirement they cover, so nobody can tell what is untested.

The QA perspective

System testing starts before the build arrives. The tester reviews the system requirements for testability, designs system tests traced to them, and agrees the environment with developers or DevOps engineers: build, configuration, data, sandboxes and accounts. The team also agrees the entry criteria, typically passed integration testing and a successful smoke test, and the exit criteria in the test plan.

During execution, record the build number, environment and test data of every run; system-level failures are hard to reproduce without them. Raise early what the environment cannot cover, such as a real payment or production-size load, and agree with the product owner how to handle that risk.

Practice exercise

Plan system tests for an online grocery order

Your task

An online grocery app is ready for system testing. Design at least one system test per requirement below, with an expected result, and decide what the system test environment must provide.

Material
R1  A customer can order up to 30 items and pay by card or cash on delivery.
R2  The customer receives an SMS when the order is out for delivery.
R3  A customer can see only their own order history.
R4  Checkout responds within 3 seconds for 95% of requests at 300 concurrent users.
R5  The app is available in Arabic and English.
Steps
  1. Classify each requirement as functional or non-functional.
  2. Write one or more system tests per requirement, including a negative test for R1: invalid input that must be rejected.
  3. List what the environment must provide: build, data, external services and accounts.
  4. Name one difference from production you would record in the report, and any question the requirements leave open.
Show the model answer
Model answer

Functional: R1 and R2. Non-functional: R3 (security), R4 (performance efficiency) and R5 (localisation, which some teams call functional).

Tests: 3 items paid with a test card in the payment sandbox → confirmed, correct total; 30 items, cash on delivery → confirmed; negative: a 31st item → refused with a clear message; the store marks the order "out for delivery" → the test phone receives an SMS with the right order number; customer A opens customer B’s order → access refused; 300 virtual users at checkout → at least 95% of responses within 3 seconds; a full order in each language → all text translated, Arabic right to left.

Environment: the release candidate deployed as in production; synthetic products and customers; payment and SMS sandboxes with test cards and phone numbers; accounts for two customers and a store operator; production’s time zone and language settings.

To record: fewer servers than production, so the R4 result is reported with the environment size. Open questions: does the limit of 30 count products or units (3 bottles of the same water)? In which language is the SMS sent?

Frequently asked questions

Who performs system testing?

Usually testers, often an independent test team, because the level compares the product with its specification and benefits from a view that is independent of the code. Developers, DevOps engineers and business analysts support it with fixes, environments and answers about requirements. On agile teams, the team’s testers usually run system tests during the sprint and before a release.

What is the difference between system testing and acceptance testing?

System testing checks the system against its specified requirements and is usually run by the delivery team; it is mainly verification. Acceptance testing checks whether the system meets the needs of its users and the business, is often run by users or customer representatives, and leads to a decision to accept it; it is mainly validation. The next lesson covers acceptance testing.

Is system testing the same as end-to-end testing?

Not exactly, though some teams use the two terms for the same thing. System testing is a test level defined by its test object, one complete system, and its test basis, the system requirements. End-to-end testing follows complete user journeys, often across several systems, and can happen during system, system integration or acceptance testing. The lesson on end-to-end testing compares them.

Can system tests be automated?

Partly. Stable checks that are repeated on every release, through the API or the user interface, and performance tests are good candidates. Exploratory sessions, usability judgements and one-off checks are usually manual. Most teams combine both.

References

Tools

Practise with Mutqan tools

Requirements → Test Cases

Convert a list of numbered requirements into traceable test cases with a requirement-to-test matrix.

Test Design

Test Scenario Generator

Generate high-level test scenarios covering functional, negative, security, usability and compatibility angles for a feature.

Test Design

Test Data Factory

One-stop factory for synthetic test data: users, customers, companies, addresses, numbers, dates and strings in JSON, CSV, SQL or XML.

Test Data

Fake User Generator

Generate fake user records (name, username, email, password, avatar seed, role) for test accounts.

Test Data

Go deeper

Your progress on this topic

Part of the QA Roadmap

Before this topic