Machine Learning Basics for Testers

Why it matters

Many ML defects never appear in the code. A model can learn from mislabelled or unrepresentative data, be evaluated on data it has already seen, or be judged by a metric that hides the errors that matter. The evaluation report then looks excellent while real users meet missed fraud, blocked customers, or a group of users served much worse than the rest.

With these basics a tester can question an evaluation instead of accepting its number: where did the test data come from, does it overlap with the training data, which errors are expensive, and how does the model do for each group of users?

Three kinds of learning, one workflow

ML work follows a workflow, which the ISTQB AI Testing (CT-AI) syllabus describes in finer steps: define the objective and acceptance criteria, collect and label data, prepare and split it, train a model, evaluate and tune it on validation data, test the final model once on test data, deploy it, then monitor it and retrain it when the data changes. Each step gives a tester something to review: the objective is a test basis, the dataset a work product, the evaluation a test result.

The kinds of learning differ in what the model learns from, which decides what a test can compare its output against:

• Supervised learning: the model learns from labelled examples, inputs paired with the correct answer (the label). Classification predicts a category (fraud or legitimate), ML regression a number (a delivery time); this statistical regression has nothing to do with regression testing. Labelled test data serves as the test oracle. The metrics in this lesson are for classification; a regression model is judged by the size of its errors, such as minutes away from the real delivery time.
• Unsupervised learning: there are no labels; the model finds structure, such as clusters of similar customers or unusual transactions. With no answer key, evaluation relies on expert review of samples, stability checks, or labelling a sample afterwards.
• Reinforcement learning: an agent learns by acting in an environment and receiving rewards or penalties, such as a robot learning warehouse routes. Tests check that the reward describes the wanted behaviour, since an agent can find unintended ways to earn it, and how the agent behaves in situations it never met in training.

Training, validation and test sets

The labelled data is divided into three sets. The model learns from the training set. The validation set is used during development to compare models, tune settings and choose the threshold. The test set (hold-out set) is kept aside and used once, after every choice is made, to estimate performance on unseen data. Some teams swap the names "validation" and "test", so ask which set a reported number comes from.

Separate sets exist because a model can fit its training data too closely. Overfitting means it has learned the noise and accidents of its sample, for example that a few merchant IDs happened to appear in fraud cases: it scores high on training data and clearly lower on validation data. Underfitting is the opposite: the model is too simple, or its inputs carry too little information, and it scores poorly on both.

A test score is honest only if the test data influenced neither training nor any choice made during development. Otherwise it is data leakage, and the score overstates what users will get. Typical forms:

• Duplicates and near-duplicates across sets: the same email forwarded twice, the same photo resized.
• The same customer on both sides, so the model recognises the customer instead of learning fraud.
• A random split of time-ordered data, which lets the model learn from the future; train on older records, test on newer ones.
• A feature (an input field the model learns from) that contains the answer, such as a chargeback date filled in only after fraud is confirmed.
• Choices made with the test set: preparation computed on all records before splitting, such as filling missing values with an average taken over every record, or tuning until the test score looks good.

Representative, clean and correctly labelled data

A model learns whatever is in its data, so reviewing the data is a test activity, like reviewing requirements. First ask whether the data represents the operating environment, the conditions the model will meet in production. Data from one region, device type, season or language, or from before a product change, may differ from what users send: this is distribution shift. After release it continues as data drift, when the inputs change (a new payment method), and concept drift, when the link between inputs and the correct answer changes (fraudsters change tactics). Ask how and when the test data was sampled, compare it with production data, and make sure the model will be monitored after release.

Then read a random sample of records yourself, not only summary statistics, and look for three kinds of defect:

• Data quality: missing values, duplicates, wrong units or formats, out-of-range values, and inconsistent categories such as "KSA", "Saudi Arabia" and "SA" in one column. Convert a CSV sample with CSV to JSON with type inference on, so that numbers stay numbers, then paste the result into the API Response Analyzer to list empty and null fields, fields whose type changes between records, and duplicate records.
• Labelling errors: wrong labels, labellers who disagree, unclear guidelines. A wrong training label teaches wrong behaviour; a wrong test label makes the metric itself wrong. Ask how disagreements were resolved, and re-check a sample.
• Data bias: under-represented groups (few Arabic-language reviews in a review classifier) or labels that repeat past human decisions (past loan approvals). The model then works worse, or unfairly, for those groups; a later lesson covers bias in depth.

Try it: CSV to JSON Try it: API Response Analyzer

The confusion matrix and its metrics

To evaluate a classifier, compare each prediction on held-out data with its label. With two classes, the class you look for is called positive (fraud) and the other negative (legitimate). Each prediction is then a true positive (TP, fraud flagged), a false negative (FN, fraud missed), a false positive (FP, legitimate payment flagged) or a true negative (TN, legitimate payment passed). The table of these four counts is the confusion matrix.

Accuracy is the share of all predictions that are correct. Precision is the share of flagged cases that are really positive: when the model raises an alarm, how often is it right? Recall is the share of real positives the model catches: how much of the problem does it find? F1, the harmonic mean of the two, is high only when both are.

Accuracy has a trap. When one class is rare, a model that always answers "legitimate" scores high accuracy and catches nothing. Below, 95% of payments are legitimate, so that useless model reaches 95%, and the real model’s 96.5% is only 1.5 percentage points better. On imbalanced data, ask for the class balance, the confusion matrix and a baseline.

Validation set: 2,000 card payments, 100 of them fraud (5%). Threshold 0.5.

                            Predicted: fraud   Predicted: legitimate
Actual: fraud (100)              TP = 60             FN = 40
Actual: legitimate (1,900)       FP = 30             TN = 1,870

Accuracy  = (TP + TN) / all     = 1,930 / 2,000 = 96.5%
Precision = TP / (TP + FP)      =    60 / 90    = 66.7%
Recall    = TP / (TP + FN)      =    60 / 100   = 60.0%
F1        = 2 x P x R / (P + R)                 = 63.2%

Baseline that always answers "legitimate": accuracy 95.0%, recall 0.0%

Try it: Percentage Calculator

Worked example: choose a threshold by cost, check each segment

The fraud model scores every payment from 0 to 1 and flags it at or above the threshold. Lowering the threshold flags more payments: recall rises or stays the same, and precision usually falls. The right threshold is a business decision based on the cost of each error type. Here the product owner estimates that a missed fraud costs 400 SAR on average and a false alarm 25 SAR in review time and customer friction.

At 0.3 the model raises four times as many false alarms as at 0.5, yet the cost of errors falls from 16,750 to 9,000 SAR, so 0.3 wins under these costs, provided the review team can handle 205 flagged payments per 2,000 instead of 90. If a false alarm blocked the payment outright, its cost would rise and the answer could change. Record the costs and the reasoning, then measure the chosen threshold once on the test set.

Then break the result down, at every threshold you compare. At 0.5, overall recall is 60%, but 40% for accounts less than 30 days old (10 of 25 fraud cases) against 66.7% for older ones: one number hid the weakest group. Compute the metrics for each segment that matters, such as language, region, device or new users, with the sample size: 25 cases is a small sample, so 40% is a signal to investigate, not a precise value.

Validation set: 2,000 payments, 100 fraud. Cost: FN = 400 SAR, FP = 25 SAR.

Threshold  TP  FN   FP     TN  Precision  Recall  Cost of errors (SAR)
0.5        60  40   30  1,870      66.7%   60.0%  40 x 400 +  30 x 25 = 16,750
0.3        85  15  120  1,780      41.5%   85.0%  15 x 400 + 120 x 25 =  9,000

Recall by segment at 0.5   Fraud cases  TP  FN  Recall
Accounts < 30 days old              25  10  15   40.0%
Accounts >= 30 days old             75  50  25   66.7%

Frequently asked questions

References

• Certified Tester AI Testing (CT-AI) v2.0ISTQB · 2026ISTQB's specialist syllabus on testing AI-based systems, covering machine-learning and generative-AI systems.
• ISO/IEC TR 29119-11:2020 — Software and systems engineering — Software testing — Part 11: Guidelines on the testing of AI-based systemsISO/IEC · 2020A technical report on the characteristics of AI-based systems that make them hard to test and on test approaches for them.
• AI Risk Management Framework (AI RMF 1.0)NIST · 2023NIST's voluntary framework for managing AI risks across the lifecycle, organised into the Govern, Map, Measure and Manage functions.