Test Isolation and Parallel Execution

Why it matters

Tests that depend on each other produce results nobody can trust. A test passes in the full run only because an earlier test created the order it reads, then fails when someone runs it alone or adds a worker; two tests edit the same user at the same moment and one of them fails one run in twenty. Such failures look like product defects, cost hours of investigation and teach a team to re-run red builds instead of reading them. Martin Fowler names lack of isolation as a common cause of non-deterministic tests.

Speed is the other half. A suite that takes an hour runs less often and reports after the developer has moved on. Parallel execution keeps a growing suite fast enough to run on every change, and it works only when tests are isolated. Isolation also helps the bug report: a test that fails alone, every time, with its own data, is a ready reproduction for the developer.

Isolation and shared-state problems

An isolated test controls its own starting state: it creates what it needs, or receives it from a fixture, and asserts only on data it created or data no test changes. Playwright's best practices ask for every test to run independently, with its own storage, cookies and data; Cypress lists tests that rely on the state of previous tests as an anti-pattern.

Shared state is anything one test can change that another test can see. Read-only data, such as a product catalogue no test edits, is safe to share; shared mutable state is not. The cure is to give each test, or at least each worker, its own copy of what it changes, and to build that state at the start of the test. Martin Fowler prefers rebuilding the starting state to relying on cleanup: when cleanup fails, a different test fails, far from the bug. Shared mutable state hides in five places:

• Server data: one account, cart, order or coupon used by many tests. One test empties the cart another is checking; a single-use coupon works only for whichever test gets there first.
• Order dependencies: test B reads the order that test A created. B passes in the usual order and fails when it runs alone, shuffled or on another worker.
• In-memory state: a module-level variable, a singleton (one object shared by the whole program), a cache, a mocked clock or a "current user" that one test sets and never resets. Worker processes do not share memory, but tests in the same worker do, and so do the parallel threads of JUnit 5.
• Browser state: cookies, local storage and a signed-in session. A saved sign-in is fine for tests that only read; a test that signs out or edits the profile needs its own session.
• Machine and environment resources: a fixed file path such as downloads/report.csv, a fixed port, one shared mailbox, a rate-limited sandbox API, a site-wide feature flag.

Try it: Fake Email Generator

How runners execute tests in parallel

In Playwright Test, Jest and pytest-xdist, a worker is a separate operating-system process that runs tests one after another. With four workers, four tests run at the same moment. Their memory is separate, but the test environment, its database, the files on disk and the network ports are not. Runners differ in what they split by default, and in how you give each worker its own resources:

• Playwright Test runs test files in parallel; the tests inside one file run in order in the same worker unless you set fullyParallel: true or test.describe.configure({ mode: 'parallel' }). Set the number with workers in the config or --workers. Every test gets a new browser context, an isolated browser profile like a fresh private window, with its own cookies and storage, and after a failure Playwright replaces the worker with a fresh process.
• pytest needs the pytest-xdist plugin: pytest -n 4, or -n auto for one worker per CPU. The worker_id fixture returns gw0, gw1 and so on.
• Jest spreads test files across worker processes and runs the tests in one file in order; --runInBand runs everything serially in one process.
• JUnit 5 runs tests sequentially in a single thread by default. Parallel execution is opt-in: the configuration parameter junit.jupiter.execution.parallel.enabled=true switches it on, but tests still run one after another until you also make them concurrent, for example with junit.jupiter.execution.parallel.mode.default=concurrent. The parallel tests are threads in one JVM, so they share static fields and singletons. @ResourceLock declares that tests use a shared resource that needs synchronised access.
• Per-worker resources: when something cannot be created per test, give each worker its own. Playwright's testInfo.parallelIndex runs from 0 to workers − 1, so worker 2 can take account 2 from a pool prepared before the run.
• Serial exceptions: a test that changes something global, such as the VAT rate or a site-wide feature flag, cannot run beside other tests. Move it into a small separate run with --workers=1, scheduled so that it never overlaps with the main run in the same environment, instead of making the whole suite serial.

Sharding: splitting a suite across machines

Workers share one machine's CPU and memory, so adding workers stops helping once the machine is busy. Sharding splits the suite across machines: each CI job runs one shard, a fixed part of the tests, and each shard can still use several workers. npx playwright test --shard=2/4 runs the second of four parts; Jest and Vitest accept the same --shard option.

Balance decides the gain, because the run ends when the slowest shard ends. Without fullyParallel, Playwright assigns whole files to shards, so one long file can keep one shard busy while the others wait; with it, individual tests are distributed. Each shard also reports only its own tests: Playwright writes a blob report per shard, and a final job merges them into one report before anyone judges the run.

Shards add an isolation risk of their own. They run on different machines but usually against the same test environment, and worker indexes repeat on every shard. A name like qa+w0@example.com, or account 0 of a pool, is therefore used by four workers at once; add the shard number, or use a random part such as a UUID.

# Four CI jobs start at the same time, one shard each
npx playwright test --shard=1/4 --reporter=blob
npx playwright test --shard=2/4 --reporter=blob
npx playwright test --shard=3/4 --reporter=blob
npx playwright test --shard=4/4 --reporter=blob

# A final job downloads the four blob reports into one folder and merges them
npx playwright merge-reports --reporter html ./all-blob-reports

Try it: UUID Generator

Worked example: three tests that fail only in parallel

A checkout suite of 120 browser tests takes 24 minutes with one worker. The team sets fullyParallel: true and four workers; the run drops to 7 minutes, but three tests now fail in some runs. The whole suite still passes every time with --workers=1, so the likely cause is interference between tests, not the product.

Every test signs in as qa.shared@example.com. "Cart total" adds items of 150.00 and 80.00 and expects 230.00, but sometimes sees 0.00, because "empty cart", running on another worker, clears the same cart. "Order history" expects the list to show three orders, the ones placed by the three ordering tests in its file; it passed only because it used to run after them in the same worker, and it fails even when run alone. "Download invoice" saves to downloads/invoice.pdf, where another worker's invoice can overwrite it.

• Each worker creates its own fresh account in a worker-scoped fixture, with a UUID in the email so shards cannot collide either.
• Tests in one worker still share that account one after another, so a beforeEach hook empties the cart before every test instead of trusting the previous one.
• "Order history" creates its own three orders through the API and asserts on their ids, not on a total count.
• The invoice is saved to test.info().outputPath(), a folder unique to each test.
• The fix is confirmed with --repeat-each=10 at four workers, because a single green run proves little.

// fixtures.ts: one fresh account per worker process
import { test as base } from '@playwright/test';
import { randomUUID } from 'node:crypto';

type Account = { id: string; email: string };

export const test = base.extend<{}, { account: Account }>({
  account: [async ({ playwright }, use) => {
    const api = await playwright.request.newContext({ baseURL: process.env.BASE_URL });
    const email = `qa+${randomUUID()}@example.com`;   // unique in every shard
    const res = await api.post('/api/test-accounts', { data: { email } });
    const account: Account = await res.json();
    await use(account);                                // shared by this worker's tests
    await api.delete(`/api/test-accounts/${account.id}`);
    await api.dispose();
  }, { scope: 'worker' }],
});

// cart.spec.ts
import { test } from './fixtures';

test.beforeEach(async ({ request, account }) => {
  await request.delete(`/api/test-accounts/${account.id}/cart`);   // start empty
});

test('download invoice', async ({ page, account }) => {
  // ...sign in as account.email and place an order...
  const downloading = page.waitForEvent('download');
  await page.getByRole('button', { name: 'Download invoice' }).click();
  await (await downloading).saveAs(test.info().outputPath('invoice.pdf'));
});

Try it: Fake User Generator

Frequently asked questions

References

• Eradicating Non-Determinism in Tests (Martin Fowler)martinfowler.com · 2011Why flaky tests destroy a suite's value, their common causes (isolation, asynchrony, remote services, time, resource leaks) and quarantine as a response.
• Playwright DocumentationMicrosoft (Playwright)Official documentation for Playwright, an end-to-end test framework that drives Chromium, WebKit and Firefox.
• Best Practices — PlaywrightMicrosoft (Playwright)Test user-visible behaviour, keep tests isolated, prefer resilient locators and web-first assertions, and debug with traces.
• Best Practices — CypressCypress.ioCypress guidance on selecting elements with dedicated attributes, avoiding fixed waits and keeping tests independent.