Automated Accessibility Testing

What automated rules can decide — and what needs a human

A rule can decide a question when the answer is fully present in the DOM, the accessibility tree and the computed styles. It cannot decide questions about meaning, quality or experience. Many WCAG success criteria mix both: a tool can see that an alt attribute exists (part of 1.1.1) but not whether the text is an equivalent of the image. Treat any published claim that tools cover a fixed share of WCAG with care; what matters is knowing which of your checks are decisive and which only raise questions.

• Usually decidable: missing alt attribute; form field or button with no accessible name; missing lang on <html>; invalid or misspelled ARIA roles, states and properties; required ARIA children or parents missing; text contrast for plain text on a solid background; zoom disabled via the viewport meta tag; empty headings and links.
• Partly decidable (tool flags, human confirms): contrast of text over images or gradients; whether an image is decorative; whether a heading structure is sensible; target size where spacing exceptions may apply; whether a live region announces at the right time.
• Needs human judgement: quality of alt text and labels; meaningful focus order and reading order; whether errors are described helpfully (3.3.3); keyboard operability of custom widgets beyond static attributes; captions and audio-description accuracy; motion that is essential or not; cognitive load and plain language; whether the whole task is achievable with a screen reader.

ACT rules: a shared definition of a test

Two tools can report different results for the same page because they interpret a criterion differently. Accessibility Conformance Testing (ACT) rules address this: each rule is a precise, documented test procedure for part of a WCAG or ARIA requirement, with its applicability, expectations, assumptions and test cases. W3C lists them on the “All ACT Rules” page; they are developed by the ACT Rules Community Group and the ACT Task Force of the Accessibility Guidelines Working Group, and they are informative — not required for determining conformance.

Rules are marked proposed or approved; an approved rule has been implemented by at least one tool and reviewed by the relevant W3C working group. Implementations are listed per tool and can be automated, semi-automated, manual or linter-based. Examples of rule titles include “Image has non-empty accessible name” and “HTML page has lang attribute”. For a tester, ACT rules are a useful tie-breaker: when a tool’s result is disputed, compare it with the rule’s expectations and test cases.

Engines and tools, described neutrally

Most tools are a rules engine plus a way to run it. Knowing which engine sits underneath explains why two tools often agree.

• axe-core — an open-source JavaScript accessibility rules engine that runs in a browser context. It returns violations, passes, incomplete results (items that need human review) and inapplicable rules, and tags rules by WCAG level. Many integrations wrap it.
• Lighthouse accessibility audit — part of Google’s Lighthouse; its accessibility checks are based on axe-core and it produces a weighted score. A score of 100 means the audited rules passed, not that the page conforms.
• Browser devtools accessibility panels — show the accessibility tree, computed name, role and states, and contrast information for a selected element. Indispensable for manual verification of automated findings.
• pa11y — an open-source command-line tool for running automated accessibility checks against URLs, commonly scripted in CI.
• Test-framework integrations — community and vendor packages add accessibility assertions to Playwright, Cypress, Jest and similar frameworks; static linters (for example for JSX) catch some issues in source code before anything renders.
• Mobile — Android documents manual testing with TalkBack and automated checks through its testing frameworks and the Accessibility Scanner app; Apple’s XCTest provides accessibility audit support in recent Xcode versions.
• Choosing — W3C’s Web Accessibility Evaluation Tools List lets you filter tools by guideline, type and environment; W3C states it does not endorse specific products.

Try it: HTML Accessibility Checker

Running checks in unit, component, E2E tests and CI

Put checks at the lowest level that can see the problem, then add broader scans higher up. Component and unit tests catch missing names and invalid ARIA on each variant of a design-system component, long before a page exists. Note that DOM emulations such as jsdom do not perform layout, and axe-core documents that its color-contrast rule does not work in JSDOM, so contrast and layout rules need a real browser. Querying elements by role and accessible name (the priority the Testing Library documentation recommends) is itself a lightweight accessibility check: if a test cannot find the button by its name, neither can a screen reader.

End-to-end tests run in real browsers and can scan pages in the states users see — after login, with a dialog open, with validation errors displayed. The Playwright documentation describes this pattern with an axe-based package; Cypress has equivalent community plugins. In CI, run the scans on every pull request for the pages and components the change touches, and a broader scheduled scan across the site.

// Example: Playwright + @axe-core/playwright, scanning a page in an error state
import { test, expect } from '@playwright/test';
import AxeBuilder from '@axe-core/playwright';

test('checkout form with errors has no WCAG A/AA violations', async ({ page }) => {
  await page.goto('/checkout');
  await page.getByRole('button', { name: 'Pay now' }).click(); // trigger validation
  await expect(page.getByRole('alert')).toBeVisible();

  const results = await new AxeBuilder({ page })
    .withTags(['wcag2a', 'wcag2aa', 'wcag21a', 'wcag21aa', 'wcag22aa'])
    .exclude('#third-party-chat') // documented exclusion, tracked separately
    .analyze();

  expect(results.violations).toEqual([]);
});

Scan the states, not just the URLs

An engine checks what is rendered at the moment it runs. A page scanned on load says nothing about the menu that opens on click, the modal, the error summary, the empty state, the dark theme or the 320 px layout. Script the interactions that reveal those states and scan after each one. Include both themes if you ship them, at least one narrow viewport, and every locale with a different direction — an Arabic right-to-left page can have different defects from its English twin.

Triaging results: false positives and false negatives

Treat each finding as a hypothesis until verified in devtools or with assistive technology. False positives happen — contrast measured against the wrong background layer, a hidden element flagged, a legitimate ARIA pattern the engine does not model. False negatives are far more common and invisible: a meaningless alt such as “image123”, a label that names the wrong field, a custom dropdown with correct roles but no keyboard support all pass the rules.

Group findings by rule and component rather than by page: one missing name in a shared button component can produce hundreds of identical results. Map each group to the WCAG success criterion, set severity by user impact on the task (a blocked checkout outranks a decorative icon), and send items marked incomplete or needs review to a person rather than ignoring them. Note that WCAG 2.2 removed 4.1.1 Parsing, so markup-validity findings such as duplicate ids are not WCAG 2.2 failures in themselves — unless they break something else, for example a label or aria-labelledby that points to the wrong element.

Try it: AI Bug Severity Suggestion

Baselining legacy issues without hiding them

Switching on a zero-violations gate for a mature product usually fails every build. A baseline makes the gate useful: record the current violations (rule id plus a stable target such as component or selector), fail the build only on findings not in the baseline, and review the baseline file in pull requests like code. Each baselined item should reference a backlog ticket so it is fixed, not forgotten; the baseline should only ever shrink. Avoid blanket rule disables — exclude a specific region with a reason and an owner, as in the example above.

{
  "generated": "2026-09-28",
  "known": [
    { "rule": "color-contrast", "target": ".legacy-footer a", "ticket": "A11Y-214" },
    { "rule": "label", "target": "#old-search input", "ticket": "A11Y-219" }
  ]
}

Reporting automated results honestly

A useful report states the scope (pages, states, viewports, themes, locales), the engine and ruleset version, the WCAG version and level targeted, the counts by success criterion and severity, new versus baselined issues, and explicitly what was not tested automatically. Never label a clean automated run as “WCAG compliant”. Write individual defects the same way as manual ones — component, observable failure, criterion, affected users — and link the rule documentation.

Try it: AI Bug Title Generator

Combining automation, manual testing and WCAG-EM sampling

A sustainable programme layers the methods: automated rules on every change; manual checks for keyboard, visual presentation and forms on every feature; assistive-technology testing of key journeys with screen readers and zoom; and, where possible, sessions with people with disabilities, which W3C’s evaluation guidance encourages throughout development.

For a formal evaluation of a whole site, the W3C WCAG Evaluation Methodology (WCAG-EM 2.0) gives the structure: define the evaluation scope, explore the product, select a representative sample, evaluate the sample, and report the findings. The sample combines a structured selection — common views, essential functionality, different types of content and technologies, and every page of each complete process — with a randomly selected set. Automated scans are well suited to exploring the whole site and choosing that sample; the sample itself is then evaluated against every applicable success criterion, largely by hand.

Try it: AI QA Checklist Generator

Frequently asked questions

References

• Evaluating Web Accessibility OverviewW3C Web Accessibility Initiative (WAI)W3C's entry point for accessibility evaluation: why to evaluate early and throughout development, and how manual checks, tools and user involvement fit together.
• All ACT RulesW3C Web Accessibility Initiative (WAI)Accessibility Conformance Testing rules: precise, documented test procedures for WCAG and ARIA requirements, used to make automated and manual results consistent.
• Web Content Accessibility Guidelines (WCAG) 2.2W3C · 2024The W3C Recommendation that defines the testable success criteria for accessible web content, organised under four principles and three conformance levels (A, AA, AAA).
• WCAG Evaluation Methodology (WCAG-EM) 2.0W3C · 2026The W3C method for evaluating a whole website against WCAG: define scope, explore, select a representative sample, audit it and report.
• Web Accessibility Evaluation Tools ListW3C Web Accessibility Initiative (WAI)A filterable list of accessibility evaluation tools by type, guideline and environment — W3C lists them without endorsing any.
• Playwright DocumentationMicrosoft (Playwright)Official documentation for Playwright, an end-to-end test framework that drives Chromium, WebKit and Firefox.
• Cypress DocumentationCypress.ioOfficial documentation for end-to-end and component testing in the browser with Cypress.
• About Queries — Testing LibraryTesting LibraryQuery types and their priority — role and label before text, test ids last — so tests find elements the way users and assistive technology do.