End-to-End Testing

Why it matters

Many serious failures live between systems, not inside them: the order is paid, but the warehouse is never told. Every component and API can pass its own tests while the customer still cannot complete a purchase. Only a test that follows the whole journey sees what the user sees.

For the business, critical journeys such as sign-up, checkout and payment are where failures cost revenue and trust directly. For the tester, E2E tests give the strongest evidence that a release is usable, and they are also the most expensive tests to build and keep running. Deciding what deserves an E2E test is a core skill.

What an end-to-end test covers

An E2E test is defined by a journey, not by a screen or a component. It starts where the user starts and ends where the user's goal is reached and visible: the order shipped, the money in the other account. Its real value is in the checks at the hand-offs, where the journey passes from one system to the next.

• Start and end: one actor, one goal, and a final outcome that can be checked.
• Systems: front end, back-end services, database, message queues and third-party services, with your own systems in the versions you plan to release.
• Checkpoints: the state after each hand-off, such as the order status, the payment reference or the email that was sent.

End-to-end testing vs system testing

In the ISTQB Foundation syllabus, system testing is a test level that focuses on the overall behaviour and capabilities of one whole system, often including functional testing of end-to-end tasks inside it. Parts it depends on may be simulated, and the interfaces with other systems and external services are the focus of a separate level, system integration testing. The ISTQB glossary calls end-to-end testing a test type, not a test level: its scope is set by a business process, which crosses every system involved, including systems your team does not own.

The two overlap, and E2E tests often run during system, system integration or acceptance testing. Some teams call any automated browser test "E2E", even one that covers a single page with the back end mocked; when you hear the term, ask what was real and what was simulated.

• Scope: one complete system, versus one business journey wherever it goes.
• Neighbouring systems: may be simulated in system testing; real, or the closest available equivalent such as a provider's sandbox, in E2E testing.
• Typical defects: wrong behaviour inside the system, versus broken hand-offs, mismatched data formats and missing steps between systems.

Worked example: an online order from cart to shipment

An online store has a web shop (front end and order service) built by your team, a payment provider in test mode, a warehouse system run by another team, and an email service. The journey: a registered customer buys two products and the order ships. One order number connects every check.

• Checks 3 to 6 in the test case below are what make it end to end: each one reads the state on the other side of a hand-off (payment to order service, order service to email, order service to warehouse, warehouse back to the web shop). With checks 1 and 2 alone, the test would only show what the customer's screen says.
• If check 5 fails while check 3 passes, suspect the hand-off from the order service to the warehouse, and record both observations in the bug report.

E2E-01  Registered customer buys two products and the order ships

Preconditions:
- customer e2e.0412@example.com exists and is verified
- product A (75.00 SAR, stock 10) and product B (40.00 SAR, stock 3)
  exist in the web shop and in the warehouse system
- payment provider in test mode
- outgoing email from the test environment goes to a test inbox

Steps and checks:
1. Sign in, add 2 x A and 1 x B to the cart.
   Check: subtotal 190.00 SAR.
2. Check out and pay with the provider's test card for an approved payment.
   Check: the confirmation page shows an order number.
3. Check: the order service has the order as PAID with the payment reference;
   stock of A is 8, stock of B is 2.
4. Check: the confirmation email with the same order number is in the test inbox.
5. Check: the warehouse system has received the order with the same 3 items.
6. In the warehouse system, mark the order as shipped.
   Check: the customer's order page shows "Shipped" and a tracking number.

Try it: User Story → Test Scenarios

Why E2E tests should be few

E2E tests sit at the top of the test pyramid, a model described in the ISTQB syllabus and in "The Practical Test Pyramid" on martinfowler.com. The pyramid pictures a healthy test suite: many small, fast, isolated tests at the bottom (component tests), fewer integration tests in the middle and only a few E2E tests at the top; models count and name the layers differently. The higher a test sits, the more it covers per run, and the slower, less isolated and costlier to maintain it is. When an E2E test fails, the cause can be in any system it passed through.

So test each rule at the lowest level where it can be tested, and keep a small set of E2E tests for the journeys that matter most to the business.

• Choose by risk: a journey earns an E2E test when its failure would stop users reaching an important goal or would cost money.
• Cover the main path plus the few alternatives that matter, such as a declined payment.
• When an E2E test finds a defect that no lower-level test caught, add a faster lower-level test for it, such as a component or API test, instead of more E2E tests.

Try it: Risk-Based Test Generator

Test data and environment challenges

Much of the effort in E2E testing goes into data and environments. The data must be consistent in several systems at once, and every run changes it: stock goes down and orders pile up. The environment must contain every system in compatible versions; shared environments are often busy or one version behind, and third-party sandboxes can be slow or unavailable.

• Create the data each test needs, with unique values per run such as a new customer email, instead of reusing records that others change.
• For payments, use the provider's sandbox and its published test cards; never real cards or real customer data.
• Wait for a condition, such as the email having arrived, with a time limit; never for a fixed number of seconds.
• Before reporting a failure, check that the environment was healthy: a sandbox outage is not a product defect.

Try it: Test Data Factory Try it: Temporary Email

Frequently asked questions

References

• ISTQB Certified Tester Foundation Level (CTFL) v4.0ISTQB · 2023The foundation syllabus of software testing: test levels and types, test design techniques, test management and tools.
• ISTQB GlossaryISTQBThe standard definitions of testing terms — test case, regression testing, severity, priority, boundary value analysis and the rest.
• The Practical Test Pyramid (Ham Vocke)martinfowler.com · 2018A widely cited explanation of how to balance unit, integration, contract and UI tests in an automated test portfolio.
• Eradicating Non-Determinism in Tests (Martin Fowler)martinfowler.com · 2011Why flaky tests destroy a suite's value, their common causes (isolation, asynchrony, remote services, time, resource leaks) and quarantine as a response.