GraphQL Testing

Why it matters

GraphQL moves risk to places REST habits do not look. Errors usually arrive inside a 200 OK response, so a status-only test passes while the operation failed. Because the client composes every request, the same data is reachable through several paths, and authorization forgotten on one of them leaks data.

One request can also ask for nested lists that cost far more than any REST call. The OWASP API Security Top 10 2023 lists unrestricted resource consumption and broken object property level authorization among its risks; GraphQL makes both easy to get wrong.

Schema, queries and mutations

The schema, written in the schema definition language (SDL), is the contract: every type, every field with its type, and the entry points. Fields of Query read data and fields of Mutation change it; ! marks a value that is never null. Arguments and variables are typed: Int is a signed 32-bit integer, so 2147483648 is refused with a request error before execution.

Compared with REST, there is usually one endpoint, the operation travels in the body, and the client, not the endpoint, chooses the returned fields. A REST API has an OpenAPI document only if the team provides one; a GraphQL server always has its schema and enforces it, so every field and argument is a ready-made test condition.

• Query: data mirrors the selection, with every requested field. A null may be a real "no value" or a failed field; only a matching entry in errors tells them apart.
• Mutation: changes data and returns the fields you select from the result; several mutation fields in one operation run in order.
• Variables: sent in a variables object and type-checked; test them like any input — missing, null, wrong type, out of range.

# Schema (SDL), shortened
type Query {
  order(id: ID!): Order
  me: User!
}
type Mutation {
  cancelOrder(id: ID!, reason: String): Order
}
type Order {
  id: ID!
  status: OrderStatus!
  total: Float!
  trackingUrl: String
  internalNote: String
  customer: User!
  items: [OrderItem!]!
}
type User { id: ID! name: String! email: String! orders(first: Int): [Order!]! }
type OrderItem { sku: String! quantity: Int! }
enum OrderStatus { PLACED SHIPPED CANCELLED }

# An operation, sent over HTTP as a JSON body:
# {"query": "query OrderSummary($id: ID!) { ... }", "variables": {"id": "1042"}}
query OrderSummary($id: ID!) {
  order(id: $id) { status total items { sku quantity } }
}

Try it: GraphQL Query Tester

Errors that arrive with HTTP 200

The specification defines the response, not the HTTP status: in practice a JSON object with data, errors or both. Where REST uses one status code for the whole request, many GraphQL servers answer 200 OK to everything they could process; some use 4xx only for requests they cannot parse or validate.

A request error — bad syntax, an unknown field, a wrongly typed variable — stops execution: errors, no data. An execution error (called a field error in earlier editions of the specification and in many tools) makes that one field null, adds an errors entry naming it in path, and still returns the rest of data: partial success. If the field is non-null, the null moves up to the nearest nullable parent. Below, the carrier service behind trackingUrl timed out.

• Assert on the body: on success errors is absent and every field has the right type; on an expected failure, check errors, path and any documented error code. extensions is defined by your server, not the specification.
• Partial success: a missing tracking link must not look like "no tracking available", and no message may expose stack traces or SQL.

HTTP/1.1 200 OK
Content-Type: application/json

{
  "errors": [
    {
      "message": "Carrier service timed out",
      "locations": [{ "line": 4, "column": 5 }],
      "path": ["order", "trackingUrl"],
      "extensions": { "code": "UPSTREAM_TIMEOUT" }
    }
  ],
  "data": {
    "order": { "status": "SHIPPED", "trackingUrl": null }
  }
}

Try it: JSON Path Tester Try it: API Response Analyzer

Query depth, size and cost

A User has orders, each Order has a customer, and that customer has orders again, so a short query can nest as deep as the client likes. Every list level multiplies the work: 100 orders, each with a customer who has 100 orders, is already 10,000 objects. A REST endpoint returns what its designers decided, usually in pages; a GraphQL request costs what the client asks for. An alias, such as o1: in front of a field, renames that field in the response, so one request can repeat the same field many times.

Expect a maximum depth, a cap on list arguments such as first, a cost budget, a time-out and rate limits that count operations, each failing with a clear entry in errors, not a crash; find the depth limit by adding one level at a time. Test only systems you own or have written permission to test, preferably a test environment, in small steps.

• With first capped at 100, send 100, 101, 0 and -1: never a silently larger page.
• In one request, send a rate-limited mutation, such as redeeming a coupon, under 10 aliases: the limit must count every alias, not the single HTTP request.

# Nesting: every list level multiplies the work
query Deep {
  me {
    orders(first: 100) {
      customer {
        orders(first: 100) {
          customer { orders(first: 100) { id } }
        }
      }
    }
  }
}

# Aliases: the same field many times in one request
query Aliases {
  o1: order(id: "1001") { total }
  o2: order(id: "1002") { total }
  # ... and so on, up to o500
}

Try it: GraphQL Query Tester

Worked example: per-field authorization

In REST, access is often checked per endpoint and method. With one GraphQL endpoint, authorization must apply every time an object or field is resolved, on every path that reaches it, whether the check sits in the resolver (the server function that produces a field's value) or in the business logic the resolver calls. Requirement: "Customers see only their own orders. internalNote is for support agents only. Only the owner may cancel an order." Customer A owns order 1042 (PLACED), customer B owns other orders, and there is one agent; each test uses that user's token.

• Tests 1 and 2: one query, two roles, two expected results.
• Tests 3 and 4 check object-level authorization (OWASP API1:2023). Re-read the order as the agent after test 4, and compare test 3 with an unknown id: a different response tells B which orders exist.
• Tests 5 and 6 reach internalNote through a nested list and a mutation result; a server that checks the rule only in the order query fails them (OWASP API3:2023, object property level authorization).

#  User        Operation, then expected result
1  Customer A  { order(id: "1042") { status internalNote } }
               -> status returned; internalNote null + error with its path
2  Agent       { order(id: "1042") { status internalNote } }
               -> both fields returned
3  Customer B  { order(id: "1042") { status } }
               -> order null; response identical to an unknown id
4  Customer B  mutation { cancelOrder(id: "1042") { status } }
               -> null + error; the agent still reads order 1042 as PLACED
5  Customer A  { me { orders(first: 5) { id internalNote } } }
               -> every internalNote null + error
6  Customer A  mutation { cancelOrder(id: "1042") { status internalNote } }
               -> status CANCELLED; internalNote null + error

Try it: API Response Comparator

Introspection: the schema describes itself

Introspection is part of the specification: system fields starting with __ return the schema. __schema lists every type and the root operation types, __type(name: "Order") describes one type, and __typename names the type of any object. Use it to list every query, mutation, field and argument to cover; ask fields for includeDeprecated: true, because deprecated fields are left out by default but still work.

Because it hands out a map of the API, the OWASP GraphQL Cheat Sheet advises disabling introspection in production and other publicly accessible environments. Exposing it can be intended for an API built for external clients; even then, the cheat sheet suggests considering disabling it for unauthenticated or unauthorized users. Disabling it does not make the schema secret: guessed fields still answer and some servers suggest names in errors ("Did you mean …?"), which the cheat sheet suggests turning off too, so authorization must hold anyway.

• Save the result as a snapshot for each release and compare snapshots: a removed field or a new required argument breaks clients.
• In each environment, send __schema without a token and with an ordinary user's token, and compare the result with the agreed policy.

query Inventory {
  __schema {
    queryType { name }
    mutationType { name }
    types {
      name
      kind
      fields(includeDeprecated: true) { name isDeprecated args { name } }
    }
  }
}

Try it: GraphQL Query Tester Try it: JSON Diff

Frequently asked questions

References

• GraphQL SpecificationGraphQL FoundationThe GraphQL query language and type system, validation rules and execution semantics.
• OWASP Cheat Sheet Series: GraphQLOWASP FoundationGraphQL-specific risks: query depth and complexity limits, introspection in production, authorization per field.
• OWASP API Security Top 10 (2023)OWASP Foundation · 2023The ten most common API weaknesses: broken object-level authorization, broken authentication, unrestricted resource consumption and more.