Test Automation Architecture

Why it matters

Without a design, a growing suite copies the same locators, URLs, waits and sign-in steps into every file, so every product change is multiplied by the number of copies. Maintenance eats the time meant for new tests, the suite falls behind the product, and the team stops trusting it.

For a tester, the architecture decides whether you can add a test in a few readable lines, switch environments without editing code, and understand a red result. You may not design the framework alone, but you will extend it, review it and live with its trade-offs.

Three layers and one rule

CTAL-TAE describes a framework in three layers, each with its own reason to change, and advises keeping the number of layers low: a layer for every purpose makes the design harder to follow.

One rule holds them together: calls go downwards only. The syllabus states that test scripts should not call the core libraries directly, so a test never drives the browser, builds a URL or reads an environment variable itself. Below the core, the chosen tool, such as Playwright, Selenium WebDriver or an HTTP library, connects to the SUT; CTAL-TAE calls this test adaptation.

• Test scripts: test cases and suites in business terms, such as checkout.payWithSavedCard() followed by an assertion on the total. They change when a requirement changes.
• Business logic: everything that knows this SUT, such as page objects, flows, API clients, test data builders and keywords. A changed screen or endpoint is fixed here, once.
• Core libraries: everything independent of the SUT, such as browser and HTTP set-up, the configuration loader, the logger and retry helpers. Any project on the same technology stack can reuse them; they change when the tools change.

shop-e2e/
  tests/                    test scripts: business terms and assertions
    delivery-fee.spec.ts
    refunds.spec.ts
  business/                 knows this SUT; calls core/, never tests/
    pages/                  page and component objects
    flows/                  user tasks across pages: signIn, placeOrder
    api/                    API clients for set-up and clean-up
    data/                   test data builders
  core/                     knows no SUT; reusable on the same stack
    config.ts               loads and validates configuration
    logger.ts               levels, redaction, run header
    browser.ts  http.ts     tool set-up, retries
  config/                   one file per environment, no secrets
    test.json  staging.json

Reusable components, and when not to share

CTAL-TAE calls tests built from a library of reusable steps and user flows structured scripting. Extract a component when the same steps appear in several tests and change for the same reason:

• Flows: one user task across several pages, such as signIn(user) or placeOrder(cart). The syllabus calls this the flow model: a second facade (a simpler interface that hides the details) over the page objects, so steps are reused across many tests.
• API set-up helpers and test data builders that create the customer or product a UI test needs, faster and more reliably than clicking through screens.
• Not worth building: a helper with ten optional flags, a base class with hundreds of methods that every page inherits, or a thick wrapper that hides the tool and makes its documentation useless. Extract on the second or third copy, not before the first.

Configuration and logging

Configuration is whatever differs between runs but not between tests: base URLs, browser, timeouts, test accounts, feature toggles. Keep one file per environment, selected by one variable such as TEST_ENV=staging, and take secrets from the CI secret store, which usually passes them to the run as environment variables, never from a committed file. Validate the configuration once at start-up in a core library, so a missing value stops the run with one clear message instead of failing 200 tests one by one.

Logging is how a failed run explains itself. CTAL-TAE describes six levels: fatal for errors that can abort the run, error when a condition or interaction fails and so does the test, warn for something unexpected that does not stop the test, info for what the test is doing, debug for details that help investigate a failure, and trace for even more detail. The syllabus also lists what the framework should record; in practice that means:

• A run header: environment, SUT build, browser and the effective configuration with secrets masked.
• Per test: name, start and end time, the main steps at info level, and the status: passed, failed, or a failure of the automation itself rather than of the SUT.
• For reproduction: ids of the records the test created and any random value or seed; request and response bodies at debug level.
• Redaction: tokens, passwords and personal data masked, for example by a pattern applied before each line is written.

// config/staging.json: committed, contains no secrets
{
  "baseUrl": "https://staging.shop.example",
  "browser": "chromium",
  "timeoutMs": 10000
}
// SHOP_API_TOKEN comes from the CI secret store; TEST_ENV=staging selects this file

// test log, info level
03:00:04 INFO  run env=staging build=4.18.2 browser=chromium token=***
03:02:11 INFO  [delivery-fee] orders of 300.00 SAR ship free: start
03:02:12 INFO  created customer C-58213 and product P-7781 via API
03:02:15 INFO  step: add P-7781 to cart, open checkout
03:02:18 ERROR delivery fee: expected 0.00, actual 7.50
03:02:18 INFO  saved screenshot and trace in results/delivery-fee/
03:02:18 INFO  [delivery-fee] end: failed after 7 s

Try it: JSON Validator Try it: JSON Schema Validator Try it: Regex Tester

Scripting approaches and their maintenance cost

CTAL-TAE lists several ways to write the tests themselves, with their advantages and drawbacks; the five below run roughly from the least up-front investment to the most. The syllabus also covers test-driven and behaviour-driven development; BDD has its own lesson later in this module.

• Capture/playback: a tool records manual actions on the SUT and generates a script. Quick to start, but nothing is reused, a recording depends on the SUT version it was captured from, and a changed screen means recording again. It fits a small scope on a rarely changing SUT.
• Linear scripting: one script per test, steps from top to bottom, no shared libraries. Easier to edit than a recording, but every change is repeated in every copy. It fits a spike or a handful of tests.
• Structured scripting: tests call reusable steps and flows from the business logic layer. It needs programming skills and an initial investment; afterwards a change is fixed in one place.
• Data-driven: structured scripts fed by rows from CSV, spreadsheet or JSON files. A new case costs one row and test analysts can add cases without code; the cost moves to managing the data.
• Keyword-driven: a test is a table of keywords named from the user's point of view, such as Add Product To Cart, with their data. It is often built on data-driven testing, and ISO/IEC/IEEE 29119-5 is devoted to it. People who do not program can compose tests, but the keyword library is demanding to design and maintain as scope grows, and a large effort for a small system.

Try it: CSV Dataset Generator

Worked example: a keyword-driven checkout suite

A retailer has two automation engineers and five testers who know the business rules but do not write code. The delivery fee rules change every quarter and the testers must add cases themselves, so the team chooses a keyword-driven framework. Robot Framework, an open-source framework written in Python, is one example of the approach; nothing here depends on it.

The testers write tables like the one below, with the boundary values of the rule "orders of 300.00 SAR or more ship free". The engineers own the keywords, which are business logic built on an API client and page objects. It fits because many people who do not program write tests in a stable domain vocabulary, and an engineer is assigned to the keywords. For two developers who already write tests in code, keywords would only add a layer to read when a test fails.

# tests/delivery_fee.robot, written by the testers
*** Settings ***
Resource    ../keywords/checkout.resource

*** Test Cases ***
Orders of 300.00 SAR ship free
    Sign In As New Customer
    Add Test Product To Cart    price=300.00
    Delivery Fee Should Be    0.00

Orders of 299.99 SAR pay 7.50 SAR
    Sign In As New Customer
    Add Test Product To Cart    price=299.99
    Delivery Fee Should Be    7.50

# keywords/checkout.resource (excerpt), maintained by the automation engineers
*** Keywords ***
Add Test Product To Cart
    [Arguments]    ${price}    ${quantity}=1
    ${sku}=    Create Product Via API    price=${price}
    Open Product Page    ${sku}
    Add To Cart    quantity=${quantity}

Frequently asked questions

References

• Certified Tester Advanced Level Test Automation Engineering (CTAL-TAE) v2.0ISTQB · 2024ISTQB's advanced syllabus on designing, implementing and maintaining test automation solutions and their architecture.
• ISTQB GlossaryISTQBThe standard definitions of testing terms — test case, regression testing, severity, priority, boundary value analysis and the rest.
• Playwright DocumentationMicrosoft (Playwright)Official documentation for Playwright, an end-to-end test framework that drives Chromium, WebKit and Firefox.