Shift-Left and Shift-Right Testing

Why it matters

A defect caught in a story review costs a conversation. After release, the same defect costs a fix, a redeploy, support time, sometimes refunds, and some of the users’ trust. Shift-left prevents defects and finds the rest while the change is still small.

But no test environment fully reproduces production: real traffic, real data, every device and every third-party service. Some failures appear only there, and without shift-right practices users notice them first. For a tester, testing stops being a phase at the end and becomes work across the whole lifecycle.

Shift left: test before and while the code is written

The ISTQB Foundation syllabus (CTFL v4.0) describes shift-left as testing earlier, without waiting for the code to be implemented or the components integrated, and stresses that later testing must not be neglected. It is the working form of the principle that early testing saves time and money. Besides the practices below, the syllabus lists CI, and better still CD, for fast feedback on every change, as in the previous lesson, and non-functional testing, such as performance, started at the component level where possible. It also names the price: extra training and effort early, expected to pay back later, and shift-left needs stakeholders who are convinced of it and support it.

• Early reviews: testers review stories, acceptance criteria and designs from a testing point of view. This is static testing, and it finds ambiguities, gaps and contradictions before any code exists.
• TDD (test-driven development): a developer writes a failing unit test, then just enough code to pass it, then refactors both, improving their structure without changing what they do.
• ATDD (acceptance test-driven development): acceptance tests are derived from the acceptance criteria before the feature is built. In BDD (behaviour-driven development), such examples are written as Given/When/Then, like the scenarios below.
• Static analysis: tools examine source code without running it and report undefined variables, unreachable code, coding-standard violations and some security weaknesses, usually on every pull request in CI.

Feature: Promo code at checkout

  Scenario: A valid percentage code reduces the total before VAT
    Given a cart totalling 150.00 SAR before VAT
    And the active code "SAVE10" gives 10% off
    When the shopper applies the code "save10"
    Then the discount is 15.00 SAR
    And the total before VAT is 135.00 SAR

  Scenario: An expired code is refused
    Given the code "WINTER25" expired yesterday
    When the shopper applies the code "WINTER25"
    Then the message "This code has expired" is shown
    And the total does not change

Try it: Acceptance Criteria Generator Try it: Requirements → Test Cases

Shift right: keep testing after the release

Shift-right is an industry term, not an ISTQB Foundation one; some teams call part of it "testing in production". Only production has real traffic, real data and every device, so the aim is to limit the damage of defects that get through and to learn what earlier testing missed, never to test on users instead of before them.

• Monitoring: dashboards and alerts on what users experience. The four golden signals from the DevOps for testers lesson, taken from Google’s SRE book, are latency (how long requests take), traffic (how much demand reaches the system), errors (the rate of failed requests) and saturation (how full its resources are). Add business signals such as orders per hour, because some defects raise no error at all.
• Canary release: the change first reaches a small share of users, and its signals are compared with a control group on the old version over the same period. If the canary is worse, it is rolled back before most users see it.
• Feature flags: new code is deployed switched off, then turned on for a group or a percentage of users without a redeploy. Test both flag states and the switch between them.
• A/B tests: two working variants run side by side on randomly split users and are judged by a metric agreed in advance. The tester checks the experiment is wired correctly; the product owner and analysts judge the result.
• Chaos experiments: the team injects a realistic fault, such as a slow dependency, to test the hypothesis that the system keeps behaving normally. Experiments start small and can be stopped at once.

Try it: API Health Checker Try it: API Response Comparator

Worked example: a promo-code field from story to production

An online store adds a promo-code field to checkout; prices are in SAR with two decimals. The figure shows the feature on both sides of the release.

• Review: the tester asks whether a code can be combined with free delivery, whether codes are case-sensitive, and whether the discount is taken before VAT and how it is rounded. The answers go into the acceptance criteria.
• ATDD and TDD: the agreed examples become acceptance tests like the scenarios above (note save10 in lower case: the team agreed that codes are not case-sensitive). The developer writes unit tests first, including 10% of 99.99 SAR, which under the agreed rounding to the nearest 0.01 must give 10.00, not 9.99.
• Static analysis: the analyser flags a query built by pasting the promo code the shopper typed into the SQL text, which would let a crafted promo code change the query itself (SQL injection). The reviewer asks for a parameterised query, which passes the promo code as data, before the merge.
• Flag and canary: the code is deployed with promo_codes off, switched on for staff, who smoke-check it with test accounts, then for 5% of shoppers. Promo errors, latency and completed checkouts are compared with the other 95% as rates, not raw counts, against rollback criteria written in advance.
• Escape: monitoring shows "invalid code" errors rising on one mobile platform, whose keyboard adds a space after the code. The fix now removes leading and trailing spaces, and the tester adds an acceptance test for codes with spaces: the finding moves left.
• Experiments: an A/B test compares a visible field with a "Have a code?" link by completed checkouts, the metric agreed in advance. A chaos experiment slows the promo service to test the hypothesis that checkout, instead of hanging, says the code cannot be checked right now and still lets the shopper pay.

Try it: Percentage Calculator

Balancing both sides

The two sides are complements, not alternatives. Left-side work is cheap and prevents defects, but it tests the team’s assumptions; right-side work sees reality, but whatever it finds has already reached someone. Set the weighting for each change by risk.

• Weight left when a failure would be severe or irreversible: payments, data migrations, privacy, security. A rollback does not refund a wrong charge.
• Weight right when only real use can answer the question: real load, the full range of devices, which design users prefer.
• Close the loop: for every escaped defect, add the missing test, review question or static-analysis rule.
• Test in production only on systems your team owns or has written permission to test, with test accounts, no real personal data and a way to stop at once.

Try it: Risk-Based Test Generator

Frequently asked questions

References

• ISTQB Certified Tester Foundation Level (CTFL) v4.0ISTQB · 2023The foundation syllabus of software testing: test levels and types, test design techniques, test management and tools.
• Canarying Releases — The Site Reliability WorkbookGoogle / O'Reilly MediaReleasing a change to a small share of traffic first and comparing it with the baseline before rolling out — deployment verification as practised in SRE.
• Monitoring Distributed Systems — Site Reliability Engineering (Rob Ewaschuk)Google / O'Reilly Media · 2016Google's SRE guidance on monitoring: black-box versus white-box monitoring and the four golden signals — latency, traffic, errors and saturation.
• Release It! Second Edition: Design and Deploy Production-Ready Software (Michael T. Nygard)The Pragmatic Bookshelf · 2018Stability patterns — timeouts, circuit breakers, bulkheads, back-pressure — and the failure modes load testing is meant to find.