Test Techniques for AI Systems

Why it matters

An AI feature evaluated once on a labelled set can still fail on inputs that set did not contain: a new phrasing, a name from an under-represented group, a crafted input, a product launched after training. Labels are expensive, so most real inputs are never checked, and the result is wrong decisions, unequal treatment and quality that drops silently after an update.

These techniques turn "there is no expected result" into test cases with a pass or fail verdict. They build on fundamentals you already have — oracles, regression testing, monitoring — and do not replace them: the system around the model still needs its functional, security and performance tests.

Metamorphic relations: an oracle without expected outputs

In metamorphic testing you run a source test case, transform its input in a controlled way to create a follow-up test case, and check a metamorphic relation (MR): how the follow-up output must relate to the source output. You do not need to know the correct output of either test.

Derive MRs from the requirement and the domain: which changes to the input must not change the result (an invariance), and which must move it in a known direction? These relations are the most used for AI systems:

• Paraphrase: reword the input with the same meaning; the decision must not change. Compare decisions — labels, fields, amounts — not wording.
• Counterfactual attribute swap: change only an attribute that must not matter, such as a name, gender or nationality; the output must be equal. This is the counterfactual pair from the lesson on safety, bias and robustness testing; written as an MR, the check runs on any input, labelled or not.
• Order invariance: change an order that carries no meaning, such as complaints within a message or documents passed as context; compared as a set, the result must be the same.
• Directional (the ISTQB AI Testing syllabus calls this monotonicity): some changes may move the result in one direction only. Adding "this is my third message about it" must not lower urgency.
• Check the transformation itself: a paraphrase that drops a fact, or a swap of an attribute that legitimately matters, produces false violations. Review the generated follow-ups, at least a sample.

Worked example: metamorphic tests for a support triage model

An online store’s model triages support messages: it returns the categories involved (refund, delivery, payment, account, other) and an urgency (low, normal, high). The requirement says urgency depends on the problem and how long it has lasted, never on who the customer is. Almost no message has a label.

The tester takes 200 recent messages as sources, derives the four MRs in the record below, and has a colleague review a sample of the generated follow-ups. The model is not deterministic, so every pair runs three times and the result is a violation rate per MR.

• MR-2 failed for S-017: changing only the signature from - Ahmed to - Priya lowered urgency to low. If a re-run repeats it, report both inputs, both outputs, the broken relation and how often it reproduced. The new name differs in gender and origin at once, so add pairs that change one signal, such as - Fatima, to narrow the cause.
• Report rates per MR and per segment, such as the language of the message: one overall rate can hide a group the model treats differently.
• Passing MRs do not prove correctness: a model that answers other and low for every message satisfies all four MRs. Keep a smaller labelled set for correctness, and use MRs for the unlabelled majority.

Model under test: support-triage v3 (output: categories, urgency)

S-017  source
  input:   I was charged twice for order 5521 and the parcel is a week late. - Ahmed
  output:  {payment, delivery}, normal

MR-1   paraphrase: categories and urgency must be equal
  input:   Order 5521 was billed two times, and the parcel is 7 days late. - Ahmed
  output:  {payment, delivery}, normal                                   pass

MR-2   attribute swap, name only: categories and urgency must be equal
  input:   I was charged twice for order 5521 and the parcel is a week late. - Priya
  output:  {payment, delivery}, low                                      FAIL

MR-3   order invariance: same set of categories, same urgency
  input:   The parcel for order 5521 is a week late and I was charged twice. - Ahmed
  output:  {delivery, payment}, normal                                   pass

MR-4   directional: urgency must not be lower than the source
  input:   S-017 + "This is my third message about it."
  output:  {payment, delivery}, high                                     pass

Run: 200 sources x 4 MRs x 3 runs; violation rate per MR, reported by language

Try it: Fake Name Generator Try it: Percentage Calculator

Comparing versions: back-to-back, shadow mode and A/B tests

Back-to-back testing runs two or more variants of a system on the same inputs and analyses every discrepancy between their outputs. For AI systems they are usually the current version (A) and a candidate (B): a new model, a changed prompt or a model retrained on fresh data. Use the labelled regression set plus recent real inputs, with identical settings.

Run A against itself first: a non-deterministic model can disagree with its own earlier outputs, and only differences beyond that baseline say something about B. Agreement is not correctness, since both versions can share a defect, and a discrepancy is not automatically a regression: classify each one as an improvement, a regression or a neutral change. Production then adds two steps:

• Shadow mode (shadow testing in the ISTQB AI Testing syllabus): B receives a copy of live inputs and its outputs are logged, never shown or acted on, while A keeps serving users. Test that the shadow path has no side effects (no messages sent, no tools called, no records written) and that its logs follow the privacy rules.
• A/B testing: a statistical approach to find which of two variants performs better. Traffic is split at random so that a share of users or tickets gets B, and a metric agreed in advance is compared, such as how often agents re-route a triaged ticket, together with guardrail metrics such as complaints. Fix the decision rule before the start too: which difference, over how many tickets, is enough. It measures the effect on users, not the correctness of single outputs.
• Keep the order: offline comparison, shadow mode, an A/B test on a small share, then full rollout. Each step exposes users to B only after the previous one gave evidence.

Try it: JSON Diff Try it: Text Diff

When inputs move: adversarial inputs, poisoning and drift

Unlike the ordinary variation of robustness testing, adversarial inputs are crafted on purpose, often with changes a person barely notices, to make a model fail: look-alike Unicode letters or invisible characters that slip past a filter, deliberate misspellings, tiny pixel changes for image models. Test them like an invariance MR: start from inputs the system handles correctly, apply each crafted change, check that the decision holds, and add every input that breaks it to the regression set. For LLM applications they are mainly prompt injection, covered in the guide on LLM security testing, and jailbreaks, which wrap a disallowed request in a role-play or split it over several messages so that the model complies, as in the lesson on safety, bias and robustness testing. Test only systems you own or have written permission to test.

Data poisoning manipulates the data a model learns from or draws on — pre-training and fine-tuning data (fine-tuning further trains an existing model on extra examples), user feedback that is fed back into training, and the documents a RAG system retrieves — so that the model behaves wrongly or with bias, or carries a backdoor: a hidden trigger that changes its behaviour only when a certain input appears. The OWASP Top 10 for LLM Applications 2025 lists it as LLM04:2025 Data and Model Poisoning. Test the process: who can change the data, whether each record’s source is kept, whether changes are reviewed before training. After each data update, a back-to-back run against the previous model can reveal unexpected changes, but a backdoor stays silent until its trigger appears, which is why the process checks matter. Never try to poison a shared or production model.

Drift, explained in the lesson on machine learning basics, makes yesterday’s evaluation stop describing today’s system. In data drift the inputs change; in concept drift the link between an input and the correct answer changes, so inputs can look the same while answers become wrong. The ISTQB AI Testing syllabus calls comparing input and output distributions with earlier ones static drift testing, and comparing outputs with fresh ground truth, such as user feedback, dynamic drift testing. These signals should trigger a re-evaluation:

• Inputs (data drift): a new mix of languages, unseen product names, missing fields, compared with the evaluation set.
• Outputs: a shifting share of each label, falling confidence, more refusals or catch-all answers.
• People and changes, where concept drift often shows first: agents re-routing tickets, complaints, a new model or prompt version, a provider updating a model whose version you have not pinned, a policy change such as a new delivery promise that changes what counts as urgent.
• Then re-run the evaluation set with a freshly labelled sample of recent inputs. A signal shows that something changed, not that quality dropped; only the evaluation shows that.

Try it: Unicode Dataset Generator

Testing transparency and explainability claims

The NIST AI Risk Management Framework treats three characteristics as distinct: transparency, how much information about the system and its outputs is available to the people using it ("what happened"); explainability, a representation of how the system works ("how" a decision was made); and interpretability, what an output means in the context of the system’s purpose ("why" it was made). In a product they appear as claims, and each claim is a requirement to test.

• Transparency: check that a promised notice ("You are chatting with an AI assistant") appears in every language and channel, and review the model documentation, such as a model card, against the release records: model version, training data, known limitations. The ISTQB AI Testing syllabus counts this review as a test activity.
• Faithfulness: turn an explanation into MRs. Remove a factor it cites, and the result or the explanation must change; change a factor it does not cite, and neither should. An explanation that survives the removal of its own reason does not describe the decision.
• Consistency and understandability: the same input gives the same explanation, and representative users can understand it, which you check as in usability testing.
• Generated reasons: when an LLM says why it answered, that text is not guaranteed to reflect how the answer was produced. Test it like any other claim; never accept it as evidence.

Frequently asked questions

References

• Certified Tester AI Testing (CT-AI) v2.0ISTQB · 2026ISTQB's specialist syllabus on testing AI-based systems, covering machine-learning and generative-AI systems.
• ISO/IEC TR 29119-11:2020 — Software and systems engineering — Software testing — Part 11: Guidelines on the testing of AI-based systemsISO/IEC · 2020A technical report on the characteristics of AI-based systems that make them hard to test and on test approaches for them.
• OWASP Top 10 for LLM Applications 2025OWASP Gen AI Security Project · 2025The ten most critical security risks of LLM applications (LLM01:2025–LLM10:2025), starting with prompt injection and sensitive information disclosure.
• AI Risk Management Framework (AI RMF 1.0)NIST · 2023NIST's voluntary framework for managing AI risks across the lifecycle, organised into the Govern, Map, Measure and Manage functions.