Environment Variables and Secrets

Why it matters

Configuration mistakes produce some of the most misleading pipeline results. A missing variable that falls back to a default can send pull-request tests to production; a wrong base URL makes a green run prove nothing about the build under test; a rotated password turns the whole suite red overnight, and someone files twenty product defects that are really one configuration change.

Secrets add a security cost. A token committed to a repository, printed in a debug line or captured in an uploaded trace can be copied by anyone who can read the repository or open the run, and test accounts on shared environments often reach real systems: payment sandboxes, e-mail providers, cloud accounts. For this reason the OWASP Secrets Management Cheat Sheet gives CI/CD pipelines a section of their own. Testers write and run these jobs, so they are part of the defence.

Recap: configuration lives outside the test code

The CI/CD testing guide set the ground rules: pipeline tests run in ephemeral environments where possible and in shared ones such as staging where realism requires it; the pipeline creates its own test data; and secrets live in the CI platform’s secret store or an external vault, scoped to the environments that need them, masked in logs, rotated, and never in the repository, never printed and never in artifacts. Workflows started by pull requests from forks get no repository secrets by default. This lesson goes one level deeper: how a value travels from the platform into a running test, and how to keep that path correct and safe.

Everything that varies between runs or targets is configuration: the address under test, the test account, timeouts, the browser, which tests to run. It stays out of the test code. The Twelve-Factor App methodology calls this strict separation of config from code and offers a litmus test: could the repository be made public right now without exposing a single credential? If not, something that belongs in the secret store is sitting in the code.

• Configuration variables: non-sensitive settings such as BASE_URL, TEST_ENV or BROWSER. They may be read, reviewed and printed.
• Secrets: values that grant access, such as a test account’s password, an API key or a database password. They are stored encrypted, masked in logs and given only to the jobs that need them.
• Platform variables: values the CI platform sets on every run, such as CI=true, the commit (GITHUB_SHA on GitHub Actions, CI_COMMIT_SHA on GitLab) and the run id (GITHUB_RUN_ID, CI_PIPELINE_ID). Stamp them on reports so every result points to the exact build and run.

Environment variables: how a test reads its configuration

An environment variable is a name–value pair that the operating system passes to a process when it starts; child processes inherit it. The CI runner sets variables for each step, the test runner inherits them, and the test code reads them: process.env.BASE_URL in JavaScript and TypeScript, os.environ["BASE_URL"] in Python, System.getenv("BASE_URL") in Java.

Values can be set at several levels. In GitHub Actions the env key works at workflow, job and step level, and the most specific wins: a step’s value overrides the job’s. Configuration variables saved in the platform’s settings (the vars context) exist at organization, repository and environment level; the environment value wins over the repository value, which wins over the organization value. GitLab offers the same idea with CI/CD variables in the project or group settings. When a test receives a surprising value, find the level it came from before suspecting the product.

• Every value is a string. HEADLESS=false arrives as the text "false", which is truthy in JavaScript; compare against the text, and convert numbers with Number().
• In JavaScript a missing variable is simply undefined, not an error: the test then calls undefined/login or, worse, a fallback address.
• Defaults are fine for harmless settings such as a timeout, never for the target or the credentials. A missing BASE_URL must stop the run, not fall back to production.
• Variables are not masked. GitHub’s documentation warns that variables render unmasked in build outputs, so a password stored as a variable is printed in clear.

// tests/config.ts: read once, check early, never print a secret
const required = ['TEST_ENV', 'BASE_URL', 'TEST_USER', 'TEST_PASSWORD'];
const missing = required.filter((name) => !process.env[name]);
if (missing.length > 0) {
  throw new Error('Missing variables: ' + missing.join(', ')); // no silent fallback
}

export const config = {
  env: process.env.TEST_ENV!,                        // staging | production
  baseUrl: process.env.BASE_URL!,
  user: process.env.TEST_USER!,
  password: process.env.TEST_PASSWORD!,             // a secret: use it, never log it
  headless: process.env.HEADLESS !== 'false',        // every value is a string
  timeoutMs: Number(process.env.TIMEOUT_MS ?? '30000'), // harmless default
  allowWrites: process.env.TEST_ENV !== 'production',
};

console.log('env=' + config.env + ' baseUrl=' + config.baseUrl); // never the password

Secrets: storage, masking and least privilege

A secret is stored once, in the CI platform’s secret store or an external vault, and injected into the job at run time, usually as an environment variable. GitHub never shows a saved secret again; you can only replace it. GitLab offers the same with its “Masked and hidden” option. Secrets never go into the repository: not in the pipeline file, a committed .env, a test data file, an exported Postman environment or a screenshot in a bug report. On your own machine, keep them in a .env file that .gitignore excludes. A secret that reached the Git history is leaked even if the next commit deletes it, because the history keeps every version: revoke and rotate it first, then clean up.

Masking replaces the exact value of a secret in the job log with *** (GitHub) or [MASKED] (GitLab). It is a safety net with holes, and both platforms say it is not guaranteed. GitHub’s documentation asks you to register every transformed value (Base64 or URL-encoded) and every derived value, such as a token signed with a secret key, as a secret too, with the ::add-mask:: workflow command, and warns that a secret wrapped in a JSON or YAML blob is much less likely to be redacted. Masking also covers only the log: traces, HAR files, screenshots, videos and HTML reports that captured an Authorization header or a typed password are uploaded as they are.

• Dedicated test credentials per environment, with the lowest role the tests need: never a person’s own account, and never an administrator “to be safe”.
• Scope each secret to the environment that needs it. In GitHub Actions an environment secret reaches only jobs that reference that environment, and with required reviewers the job cannot read it until someone approves; GitLab’s protected variables reach only pipelines on protected branches or tags.
• Give the pipeline’s own token read-only access by default (permissions: contents: read for GITHUB_TOKEN) and widen it only in the job that needs more.
• Keep the fork rule: workflows started by pull requests from forks get no repository secrets. Run secret-dependent tests after merge instead of working around it.
• Prefer short-lived credentials to long-lived keys: with OpenID Connect (OIDC), the job shows the cloud provider a token issued by the CI platform and receives short-lived credentials for that run, so no cloud key is stored as a secret. Rotate the rest on a schedule and immediately after any exposure, and delete the ones no job uses.

Try it: Random Secret Generator Try it: Base64 Encoder Try it: API Authentication Tester

Per-environment configuration: one suite, many targets

The same suite should run against an ephemeral pull-request environment, staging and production with no code change: only configuration differs. Agree with the team, in writing, what changes per environment and what must not. The test code and the build under test stay the same; a common practice is to build once and promote that one build through the environments, so a pass on staging says something about the build that reaches production.

Two mechanisms carry the differences. Non-secret settings can live in a reviewed, committed file per environment (config/staging.json) or in environment-level variables; secrets come only from the store, scoped per environment. GitHub Actions has environments with their own variables, secrets and protection rules; in GitLab a project variable can be given an environment scope (scoping group variables needs a paid tier). Production deserves its own rules: a small read-only smoke set, a dedicated monitoring account and a guard in the suite that refuses to run data-changing tests there.

Setting           PR run (ephemeral)    staging                 production
----------------  --------------------  ----------------------  ----------------------
BASE_URL          localhost:3000        staging.shop.example    shop.example
Test account      seeded per run        qa-buyer (staging)      smoke-monitor (read)
Payments          mocked                provider sandbox        not exercised
Tests that run    @pr subset            full regression         @smoke only
Writes data       yes, reset per run    yes, own records        no
Secrets from      none needed           staging environment     production environment
Approval          none                  none                    required reviewer

Try it: Text Diff

Worked example: staging tests and a production smoke check

An online shop deploys from main. The team wants the full end-to-end suite on staging after each merge and, once the release reaches production, a smoke subset with a read-only account. The excerpt below shows the test jobs; build and deploy jobs are omitted.

• Same names, different values: vars.BASE_URL and secrets.TEST_PASSWORD resolve to the staging or the production value depending on the job’s environment, because an environment-level value overrides a repository-level one with the same name. The test code does not change.
• The workflow runs on pushes to main only; pull requests use the ephemeral setup from the table above and need no shared-environment secrets.
• permissions: contents: read keeps GITHUB_TOKEN read-only in both jobs.
• The production job waits for a required reviewer (set on the environment in the repository settings) and only then can read production secrets; it runs the @smoke tests with a read-only account, and allowWrites in the config is false.
• In the log, the Run npx playwright test group lists the step’s env values: BASE_URL in clear and TEST_PASSWORD as ***. The tests then print env=staging baseUrl=https://staging.shop.example. When a run fails, these lines tell you what was actually tested — the next lesson builds on them.

# .github/workflows/e2e.yml (excerpt; build and deploy jobs omitted)
on:
  push:
    branches: [main]
permissions:
  contents: read                       # GITHUB_TOKEN can only read the code
jobs:
  e2e-staging:
    runs-on: ubuntu-latest
    environment: staging               # staging variables and secrets only
    steps:
      - uses: actions/checkout@v7
      - run: npm ci
      - run: npx playwright install --with-deps
      - run: npx playwright test
        env:
          TEST_ENV: staging
          BASE_URL: ${{ vars.BASE_URL }}                # variable: shown in logs
          TEST_USER: ${{ vars.TEST_USER }}
          TEST_PASSWORD: ${{ secrets.TEST_PASSWORD }}   # secret: masked in logs
  smoke-production:
    needs: e2e-staging                 # full file: after the production deploy job
    runs-on: ubuntu-latest
    environment: production            # waits for a required reviewer
    steps:
      - uses: actions/checkout@v7
      - run: npm ci
      - run: npx playwright install --with-deps
      - run: npx playwright test --grep @smoke
        env:
          TEST_ENV: production
          BASE_URL: ${{ vars.BASE_URL }}
          TEST_USER: ${{ vars.TEST_USER }}
          TEST_PASSWORD: ${{ secrets.TEST_PASSWORD }}

Try it: YAML Formatter

Frequently asked questions

References

• OWASP Cheat Sheet Series: Secrets ManagementOWASP FoundationHow API keys and secrets should be generated, stored, rotated and kept out of code and logs.
• Using secrets in GitHub ActionsGitHubStoring credentials as repository, environment or organization secrets, how GitHub masks them in logs, and why they are not passed to workflows triggered from forks.
• GitHub Actions documentationGitHubHow workflows, jobs, runners and artifacts are defined in GitHub Actions — where automated tests run on each push or pull request.
• GitLab CI/CDGitLabHow pipelines, jobs and artifacts are defined — where an automated regression suite runs on every change.