Exploratory Testing

What exploratory testing is — and what it is not

The term is associated with Cem Kaner, and the short definition most practitioners quote comes from James Bach: simultaneous learning, test design and test execution. In scripted testing those three happen at different times and often by different people — someone analyses the requirement, someone writes the steps, someone else follows them later. In exploratory testing they happen in one head, in a loop of minutes: you try something, observe the result, form a new question and design the next test from the answer.

The ISTQB Foundation Level syllabus (CTFL v4.0) lists exploratory testing among the experience-based test techniques, next to error guessing and checklist-based testing, and notes that it is sometimes structured as session-based testing: a time-boxed session guided by a test charter that states the test objectives. That structure is what separates exploratory testing from ad hoc testing. Both are unscripted; only one is accountable.

• It is guided — by a charter (the mission of the session), by the risks of the feature and by heuristics that generate test ideas.
• It is recorded — actions, data, observations, questions and bugs are noted as you go, so the session can be reported and reviewed.
• It is skilled — its value depends on the tester’s product knowledge, technique and judgement; that dependence is both its strength and its main risk.
• It still uses test design — good explorers apply boundaries, partitions and state models just like test case writers do, only in real time instead of up front.

Exploratory vs scripted testing: a fair comparison

Scripted and exploratory testing are not rival camps; they answer different questions. A scripted test checks a known expectation the same way every time, which makes it repeatable, reviewable, easy to delegate and easy to automate. An exploratory session looks for what nobody anticipated — the misunderstood requirement, the odd interaction between two features, the error message that exposes an internal code — which a script cannot find, because a script only covers what its author thought of.

In practice most testing sits on a continuum. A test case with a vague step (“check the report looks right”) already leaves room for exploration; a charter that says “explore refunds for partially shipped orders” already constrains it. The useful question is not which approach is better but how much of each a given feature, risk level and team needs.

• Repeatability — scripted: high, the same steps and data every run. Exploratory: deliberately low; each session varies, which is why it keeps finding new problems.
• Finding new defects — scripted: limited to what was foreseen. Exploratory: strong, especially in new or poorly specified features.
• Evidence for audits and traceability — scripted: strong, each case links to a requirement. Exploratory: adequate only when charters, notes and debriefs are kept.
• Dependence on the tester — scripted: lower at execution time, because the skill went into the design. Exploratory: high throughout.
• Cost of change — scripted: every requirement change means editing cases. Exploratory: charters adapt at the next session.
• Automation — scripted checks with clear expected results are natural automation candidates; exploration itself cannot be automated, but it produces the checks worth automating.

When exploratory testing pays off

Use exploratory testing where uncertainty is high and scripted coverage would be thin, late or blind to the real risk. Scripted testing is the better fit where the same check must be repeated exactly, where a regulator or customer requires traceable evidence per requirement, or where testers are new to a domain and need written guidance.

• New features — before test cases exist or while the design is still moving, exploration teaches you the feature and produces the first cases.
• Vague or incomplete requirements — a short session surfaces questions for the product owner faster than reading the document again.
• High-risk areas — payments, authentication, permissions, data migration — as a complement to scripted and automated checks, never as a replacement.
• After a bug fix — explore around the fix: the same defect on neighbouring paths, and side effects of the change.
• Integration points and error handling — timeouts, retries, partial failures and third-party responses that specifications rarely describe.

Try it: Test Scenario Generator

Heuristics: how explorers generate test ideas

A heuristic is a fallible rule of thumb — in testing, a prompt that suggests what to try next. Heuristics do not guarantee a bug; they stop you running out of ideas and stop you testing only the path you already know works. Elisabeth Hendrickson’s Explore It! is organised largely around such variations. The families below are described in our own words and are a starting point, not a complete catalogue.

• Vary the inputs — empty, very long, boundary values, wrong type, leading and trailing spaces, special characters, Arabic and mixed-direction text, emoji, pasted content, and values in other formats (dates, phone numbers, decimals with a comma).
• Vary the states — the same action as a new user, a user whose session expired, a locked account, an order that is already paid, a cart changed in another tab.
• Vary the sequences — do steps out of order, repeat them, skip one, go back and forward, submit twice, cancel after saving.
• Interrupt — drop the network mid-request, close the tab, put the app in the background, let the session time out, rotate the device, take a call during payment.
• Vary the data lifecycle — create, read, update and delete the same record from different places; delete something other records refer to; edit a record someone else is viewing.
• Vary the configuration — browser, device, language, locale and time zone, screen size and zoom, permissions granted or denied, feature flags on and off.
• Vary timing and volume — act very fast and very slowly, near midnight or at month end, with one item and with thousands.
• Follow the data — enter a value and trace everywhere it appears: confirmation page, email, PDF, admin screen, export file, API response.
• Take a persona — explore as an impatient user, a first-time user, an administrator, a screen reader user or an attacker.

Try it: Special Character Dataset Generator Try it: Unicode Dataset Generator Try it: Boundary Value Generator Try it: State Transition Generator

Oracles: how you recognise a problem

An oracle is the means by which you recognise a problem; the ISTQB glossary defines a test oracle as a source to determine an expected result to compare with the actual result. A scripted test carries its oracle in the expected-result column. An explorer usually has no written expected result for most of what they see, so they reason from consistency: behaviour that is inconsistent with something the product ought to be consistent with is a candidate problem.

Oracles are heuristic too. An inconsistency is a reason to investigate, not proof of a bug — some differences are intended. The tester’s job is to notice, gather evidence and explain why it might matter; the product owner decides whether it is a defect.

• Consistency within the product — the same concept named or calculated differently on two screens; a cart total that differs from the invoice total.
• Consistency with its history — it behaved differently in the previous release and nobody announced a change.
• Consistency with claims — the requirement, help text, API documentation, release notes or what was agreed in refinement.
• Consistency with comparable products — similar products or established platform conventions behave differently for no stated reason.
• Consistency with purpose and user expectations — the feature technically works but does not let the user do what it exists for.
• Consistency with standards and regulations — accessibility guidelines, data-protection rules, local conventions for dates, numbers and currency.
• Consistency with the image the organisation wants — blaming or rude error messages, broken Arabic text, an internal error code shown to a customer.
• Explainability — if you cannot explain the behaviour to a colleague, that alone is worth a question.

Try it: API Response Comparator Try it: Text Diff

Test tours: exploring with a tourist’s map

James Whittaker’s book Exploratory Software Testing builds its approach around a tourist metaphor: treat the product like a city a tourist visits, and pick a tour with a theme that decides where you go and what you look at. A tour gives a session a shape without writing steps, and it makes it easy to divide exploration among several testers so that everyone does not walk the same streets.

The book describes a larger set of tours grouped into “districts” of the product. The ideas below are paraphrased, with the book’s names where we use them; they are examples, not a complete or official list, and many teams invent tours for their own domain.

• Guidebook tour — follow the user manual, help pages or onboarding exactly and check the product does what they say.
• Money tour — exercise the features that sell the product or earn the revenue, the ones shown in demos.
• Landmark tour — pick the key features and visit them in different orders, checking the journey between them.
• FedEx tour — take one piece of data (an address, an order) and follow it through every screen, email, report and integration that touches it.
• Back-alley tour — visit the least-used features and settings, where testing is usually thinnest.
• Saboteur tour — deny the application what it needs: network, storage, permissions, a dependency that responds slowly.
• Obsessive-compulsive tour — repeat the same action again and again: double submissions, reopening the same dialog, adding the same item many times.
• Supermodel tour — look only at the surface: layout, truncation, alignment, contrast, right-to-left mirroring, text in every language.

Worked example: exploring a login with a one-time password

For example, suppose a fictional banking web app adds a second login step: after the password, the user receives a six-digit one-time password (OTP) by SMS, valid for a few minutes, with a limited number of attempts and a “Resend code” link. The acceptance criteria cover the happy path, a wrong code and an expired code. The tester has half a day. Instead of one long unfocused session, they write a handful of charters in the form “Explore <target> with <resources> to discover <information>” and rank them by risk:

During charter 2 the tester applies the sequence and interruption heuristics: clicks “Resend code” five times quickly, then enters the first code received. Using the oracle “the product should agree with its own messages”, they notice that the screen says previous codes are no longer valid while the first code is accepted. The note becomes a bug report; the idea becomes a regression check. The titles below illustrate the kind of findings such sessions produce — they are examples, not results from a real product.

• “Previously sent OTP is still accepted after Resend, contradicting the on-screen message”
• “Failed-attempt counter resets when the OTP page is reloaded, bypassing the attempt limit”
• “OTP field rejects a code pasted with a trailing space from the SMS app”
• “Arabic-Indic digits typed on an Arabic keyboard are not accepted in the OTP field”
• “Countdown shows 00:00 but the code is still accepted for about a minute”

Charter 1  Explore OTP entry with wrong, partial, pasted and malformed codes
           to discover how attempts are counted and when the account locks.
Charter 2  Explore "Resend code" with rapid repeated clicks and a slow network
           to discover which code is valid and whether rate limits apply.
Charter 3  Explore OTP expiry with an idle session, backgrounding and page reloads
           to discover timing gaps between the countdown and the server.
Charter 4  Explore the OTP step in two tabs and on two devices at once
           to discover session and state conflicts.
Charter 5  Explore OTP screens and SMS text in Arabic and English, RTL, zoom
           and a screen reader to discover usability and localisation problems.

Try it: AI Edge Case Generator Try it: AI Negative Scenario Generator

From notes to bug reports and regression cases

Notes are what make exploration reviewable. Write them while you test, not afterwards: a timestamp, what you tried, the data you used, what you observed, your questions and anything that looked wrong. Short factual lines are enough. A screen recording, a captured request or HAR file and the console log save a lot of reconstruction later.

At the end of a session, sort the notes into three piles. Bugs become bug reports: reproduce each once more, reduce the steps and write it in the team’s format (the bug report guide covers the fields, and severity versus priority). Questions go to the product owner or a developer. Ideas that deserve to be checked on every release become regression cases: written as normal test cases with a precise expected result, with the ones worth automating marked.

• Record the charter, build and environment with the notes so a later reader knows what was and was not covered.
• Log what you could not test and why (blocked environment, missing data) — it matters as much as what you tested.

NOTE  10:42  Resend x5 fast, entered 1st code (481207) -> accepted.
             Screen says "Previous codes are no longer valid". Bug?
NOTE  10:46  Repeated with 2 resends: same. Staging build 3.8.1, Chrome 129.

BUG   Earlier OTP is still accepted after a new code is sent via Resend
      Steps:    1) Log in as layla.h@example.com  2) Wait for OTP A
                3) Click "Resend code", receive OTP B  4) Enter OTP A
      Expected: "Code expired" error (on-screen text and acceptance criterion 4)
      Actual:   Login succeeds with OTP A

REGRESSION CASE  TC-OTP-017  Only the latest OTP is valid after Resend
      Expected: earlier codes rejected with "Code expired"; attempt is counted
      Automate: yes, at API level (verify endpoint called with a superseded code)

Try it: AI Bug Description Generator Try it: AI Bug Title Generator Try it: Test Case Generator

How exploratory testing and automation work together

Automation and exploration have complementary jobs. Automated checks confirm, quickly and on every change, that known behaviour still works, and they free testers from repeating the same regression by hand. Exploration spends that freed time where automation is blind: new behaviour, unexpected interactions and judgements about whether the product is actually useful. The general division of labour between manual and automated testing, and the test pyramid, are covered in the manual versus automated testing guide; the Agile Testing quadrants likewise place exploratory testing among the business-facing tests that critique the product.

• Exploration feeds automation — every bug found in a session is a candidate regression check; automate the ones with a clear, stable expected result, preferably at API or unit level.
• Automation feeds exploration — a new failure, a flaky test or a gap in a coverage report is a good charter.
• Scripts as exploration aids — use code or data generators to reset state, create a thousand orders or reach a hard-to-reach state, then explore by hand from there.
• Explore the blind spots — visual layout, localisation, accessibility with real assistive technology and error recovery rarely have strong automated oracles.
• Put it in the workflow — for example, a chartered session on each risky story once its build reaches a test environment, before the story is marked done.

Try it: AI Regression Test Generator Try it: Regression Scenario Generator Try it: Test Data Factory

Common mistakes

• Treating exploratory testing as unplanned testing — no charter, no notes, no debrief, and afterwards nobody can say what was covered.
• Charters that are too broad (“explore the app”) or so narrow that they are test scripts in disguise.
• Exploring only the happy path with valid data, which repeats what the scripted tests already do.
• Reporting a bug without isolating it — a few minutes finding the trigger saves a day of back-and-forth.
• Replacing regression checks with exploration, or exploration with regression checks; each finds what the other misses.
• Judging testers by bugs per session, which rewards trivial reports and punishes exploring areas that turn out to be solid.

Frequently asked questions

References

• ISTQB Certified Tester Foundation Level (CTFL) v4.0ISTQB · 2023The foundation syllabus of software testing: test levels and types, test design techniques, test management and tools.
• ISTQB GlossaryISTQBThe standard definitions of testing terms — test case, regression testing, severity, priority, boundary value analysis and the rest.
• Explore It! Reduce Risk and Increase Confidence with Exploratory Testing (Elisabeth Hendrickson)The Pragmatic Bookshelf · 2013The discipline of exploratory testing: charters, heuristics for varying inputs and states, and how it complements scripted tests.
• Exploratory Software Testing: Tips, Tricks, Tours, and Techniques to Guide Test Design (James A. Whittaker)Addison-Wesley (Pearson) · 2009The book that organised exploratory testing around "tours" — structured ways to explore features, data and user journeys.
• Session-Based Test Management (Jonathan Bach)Satisfice, Inc. · 2000The original description of session-based test management: chartered, time-boxed exploratory sessions, session sheets and debriefs.
• Lessons Learned in Software Testing: A Context-Driven Approach (Cem Kaner, James Bach, Bret Pettichord)Wiley · 2001Practical lessons on bug reporting, test design, automation and managing testing — the context-driven school.
• Agile Testing: A Practical Guide for Testers and Agile Teams (Lisa Crispin, Janet Gregory)Addison-Wesley (Pearson) · 2009The agile testing quadrants and how testers work inside a delivery team — the basis of "testing at the right layer".