LLM Fundamentals for Testers

Why it matters

Each mechanism is a source of defects. Sampling makes the same input give different answers; the context window decides what the model can see; output limits cut answers off; a system prompt is text, not an enforced rule; and a tool call turns generated text into an action. A tester who does not know this either writes exact-match tests that fail at random or accepts "the AI is like that" and lets real defects through.

Knowing the fundamentals lets you predict where an LLM feature will break, design tests for those places and write bug reports a developer can reproduce. The model is still a component under test, and every QA fundamental still applies.

Tokens and the context window

A tokenizer splits all text into tokens: common words, pieces of longer words, fragments of numbers, punctuation. Tokenizers differ between model families, so the same text can give different counts on different models, and a sentence in Arabic or Hindi can take more tokens than in English. Limits and cost are counted in tokens, and response time grows with the tokens generated, so use the provider’s token counter or the usage the API reports instead of estimating from characters.

The context window is the maximum number of tokens the model can handle in one request, usually input and output together. Everything the model knows about the situation must fit in it: the system prompt, tool definitions, the conversation, retrieved documents, tool results and room for the answer. The model remembers nothing between requests; a chat seems to remember only because the earlier messages are sent again with each request, usually by the application.

When a conversation outgrows the window, the application trims or summarises it, or the request fails, and an instruction or order number from the start can be lost. The maximum output tokens is a separate limit: an answer that reaches it is cut off, often mid-sentence, and the response’s stop reason says so. Both limits are boundaries, so apply boundary value analysis to them, with long Arabic and Hindi input too; generated filler text measured with the String Length Counter helps you grow an input step by step.

Illustrative split (every tokenizer differs):
  "Where is my order 5521?"  ->  Where| is| my| order| 55|21|?   (7 tokens)

Context budget for one request (example model with a 16,000-token window)
  System prompt                            1,200
  Tool definitions (3 tools)                 900
  Conversation so far (18 messages)       11,500
  New customer message                       400
  Reserved for the answer (max output)     2,000
  Total                                   16,000   <- window full

Try it: String Length Counter Try it: Lorem Ipsum Generator

How the next token is chosen: temperature and sampling

At each step the model gives every token in its vocabulary a probability of coming next. Always taking the most probable one is greedy decoding; many applications sample instead, picking at random in proportion to the probabilities, so answers vary from run to run.

Temperature rescales those probabilities: below 1 it favours the most likely token, above 1 it gives unlikely tokens more chances. Top-p (nucleus sampling) keeps the smallest set of most likely tokens whose probabilities add up to at least p; top-k keeps the k most likely. Not every model accepts every parameter, so record the values actually sent.

In the example below the store’s policy is 30 days, yet a wrong number is chosen in about one run in four at temperature 1.0 and four in ten at 2.0, and a long answer makes hundreds of such choices. A low temperature reduces variation but in practice does not guarantee identical output. Test at the production settings, run important cases several times and compare the runs, for example with Text Diff.

Context: "... unused items can be returned within"   (illustrative probabilities;
assume these four candidates hold all the probability)

Next token       temperature 0.5   temperature 1.0   temperature 2.0
" 30"                      91.5%             70.0%             48.6%
" 14"                       7.5%             20.0%             26.0%
" 7"                        0.9%              7.0%             15.4%
" thirty"                   0.2%              3.0%             10.1%

Wrong number (14 or 7)      8.4%             27.0%             41.3%

Try it: Text Diff

System prompts and tool use

A request is a list of messages with roles. The system prompt (some providers say system or developer message) holds the application’s instructions: role, rules, scope, language and format. Models are trained to give it priority, but it is not enforced: it is text in the same context as the user’s input. The model can fail to follow it, user text or a document can try to override it (prompt injection, LLM01:2025 in the OWASP Top 10 for LLM Applications), and it can leak (System Prompt Leakage, LLM07:2025). Keep secrets out of it, and enforce permissions, prices and limits in code.

With tool use, or function calling, the application describes each tool with a name, a description and a JSON Schema for its arguments, a document that lists the fields, types and values the arguments may contain and is itself written in JSON, the plain-text format of named fields and values, such as {"order_id": "5521"}, that most web APIs exchange. The model never runs anything: it returns a structured request to call a tool with arguments it generated, and the application validates them, runs the tool and sends the result back in a new request. Treat generated arguments as untrusted input; they can be malformed, or valid but wrong, such as an order number the customer never gave. The JSON Schema Validator checks logged arguments against the schema. Tools that change things are where excessive agency, LLM06:2025, turns a wrong answer into a wrong action.

Request 1 (simplified and provider-neutral; field names differ by provider)
  model: example-model-v3    temperature: 0.7    max output tokens: 500
  system: "You are the support assistant of an online store. Answer in the
           customer's language. Hand every refund request to a human agent."
  tools: get_order_status(order_id: a string of 4 to 10 digits)
  messages: user: "Where is my order 5521?"

Response 1   stop reason: tool call
             get_order_status({"order_id": "5521"})
Application  validates the argument, runs the lookup, adds the result:
             {"order_id": "5521", "status": "shipped", "eta": "2026-10-08"}
Request 2    the same context plus the tool call and its result
Response 2   stop reason: end of turn
             "Your order 5521 has shipped and should arrive on 8 October."

Try it: JSON Schema Validator

Typical failure modes

Most LLM defects fall into a few families, each traceable to a mechanism above; naming the mechanism points to the fix and to the regression test.

• Hallucination: fluent, confident content that is false or unsupported, such as an invented policy or order status (NIST AI 600-1 calls it confabulation). The model produces likely text, not checked facts, so test questions whose answer is not available and expect "I don’t know" or a hand-off.
• Instruction drift: wrong language, wrong format or out-of-scope advice, often late in long conversations. Refusing a legitimate request is a failure too.
• Inconsistency: the same input is approved in one run and refused in another; for a decision that must be stable, the flip itself is the defect.
• Truncation and lost context: an answer cut off at the output limit with unclosed JSON, or a fact gone after history trimming.
• Malformed structured output: invalid JSON, missing or extra fields, or prose around the JSON that breaks the parser.
• Tool-use errors: a wrong or missing tool call, invented arguments, a tool error covered by an invented result, loops, or an action without the required confirmation.
• Injection and leakage: text from a user, a page or a document overrides the instructions or extracts the system prompt; the LLM security testing guide covers benign tests for these.

Worked example: tracing one wrong answer to its cause

A customer complains that the support assistant promised his second order "tomorrow", although it had not shipped. Instead of reporting "the bot invents dates", the tester reads the logged request for that turn as it was actually sent.

It is message 31, "And the second order?", and the response has no tool call. The order number was given in message 3, but the application keeps only the last 20 messages, so the model had no value for the required order_id argument and, instead of asking, wrote a likely-sounding answer. At temperature 0.7, re-sending the logged request five times gave an invented date four times and a question for the order number once.

• Bug report: model and prompt version, parameters, the trimmed request, the stop reason, "invented date in 4 of 5 runs", and the cause: trimming dropped the order number and the prompt has no rule for a missing one.
• New tests: a 30-message conversation with the order number only in message 3, and a question with no order number at all. Expected in every run: the assistant looks the order up or asks for the number, and never states a date without a tool result.
• Question for the team: should key facts such as order numbers survive history trimming?

Frequently asked questions

References

• Certified Tester AI Testing (CT-AI) v2.0ISTQB · 2026ISTQB's specialist syllabus on testing AI-based systems, covering machine-learning and generative-AI systems.
• Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (NIST AI 600-1)NIST · 2024The generative-AI profile of the AI RMF: risks specific to generative models, such as confabulation and information integrity, and suggested actions to manage them.
• OWASP Top 10 for LLM Applications 2025OWASP Gen AI Security Project · 2025The ten most critical security risks of LLM applications (LLM01:2025–LLM10:2025), starting with prompt injection and sensitive information disclosure.
• Certified Tester Specialist Level – Testing with Generative AI (CT-GenAI)ISTQBISTQB's syllabus on using large language models across the test process — the other direction from testing AI systems.