Continuous Integration, Delivery and Deployment

Why it matters

Integrating and releasing rarely are both risky. When weeks of work from several people meet at once, conflicts and integration defects pile up and nobody can tell which change caused which failure. These practices replace big, rare events with small, frequent ones, each checked while its author still remembers it.

For a tester, the practice decides what the pipeline already proves and whether any person checks a change before users see it. Knowing which one your team really follows tells you which risks still need you.

Continuous integration: merge often, verify every merge

Martin Fowler’s widely cited description defines CI as a practice in which every team member merges their changes into a shared codebase at least daily, and each integration is verified by an automated build that includes tests. A conflict between two changes made this morning is quick to fix; found three weeks later, it is not.

CI is a team habit supported by a tool, not the tool itself. A CI server that builds a long-lived branch once a week, or a main branch left red for days while people keep merging, is a CI tool without continuous integration. The habits that make it work:

• One shared mainline in version control, with everything needed to build and test the product.
• A self-testing build, triggered by every push to the mainline: it runs the automated tests and fails when any of them fails. Many teams also build each pull request before it is merged.
• A build measured in minutes, so developers wait for the verdict instead of moving on.
• A broken build is fixed first: nobody adds changes on top of a red mainline.

Continuous delivery and continuous deployment

Continuous delivery extends CI beyond the merge. The pipeline builds the product once, deploys that same build to production-like test environments and runs further checks there, such as API, end-to-end and non-functional tests. Every build that passes is a release candidate that could go to production at the push of a button; when it actually goes is a business decision.

Continuous deployment removes that decision: every change that passes all automated stages is deployed to production automatically. It therefore requires continuous delivery, which in turn requires CI.

Two terms cause confusion. "CD" is used for both, so when a team says it does "CI/CD", ask which one it means. And some teams separate deployment (the new version runs in production) from release (users can see the feature), for example by keeping a finished feature switched off until a planned date.

The role of automated tests in each practice

In all three practices, automated tests are the evidence the pipeline acts on; without them, CI only proves that the code builds. What changes is how much depends on that evidence. The ISTQB Foundation syllabus adds that even with the high level of automation in DevOps, manual testing from the user’s perspective is still needed.

• CI: the tests decide whether a change may join the mainline, so they must be fast: mostly static checks, unit and component tests.
• Continuous delivery: the tests decide whether a build is releasable. Slower API, end-to-end and non-functional tests run later, only on builds that passed the fast stages, and people can still explore the release candidate before anyone presses the button.
• Continuous deployment: the tests are the only gate before users. A risk the pipeline does not check is not checked at all, so post-deployment checks, monitoring and a fast rollback complete the safety net.
• In every practice, a flaky test, one that fails at random, damages the safety net: the team learns to re-run and ignore red builds, and real failures slip through.

Try it: Regression Scenario Generator

Worked example: one change, three practices

An online pharmacy lowers its free delivery threshold from 200.00 SAR to 150.00 SAR. The log shows one pipeline run; this part is identical under all three practices.

• CI: by 09:52 the change was merged, and unit tests had checked 149.99 and 150.00 at the new threshold, and 199.99, a total that paid a delivery fee before the change and now ships free. A failure at 09:44 would have kept it out of main while the change was still fresh.
• Continuous delivery: build 2.14.0+512 waits on staging. The tester explores the cart (what does the free delivery banner say now? what about exactly 150.00 SAR with a discount code?) and compares the cart API response on staging with production. The product owner releases it at 14:00 with the weekly promotion.
• Continuous deployment: the same build reaches production at about 10:12 with nobody deciding. Post-deployment checks call the health endpoints and smoke-test the cart; if they fail, the team rolls back. The exploratory questions had to be asked before the merge or turned into automated checks.
• Had the end-to-end stage failed at 10:09, no release candidate would exist under either practice: the pipeline stops the change, not a person.

Change: free delivery from 150.00 SAR (was 200.00 SAR)

09:40  pull request #418 opened
09:41  build .......................................... passed
09:44  static checks + unit tests ..................... passed
09:49  API tests against a temporary database ......... passed
09:52  merged to main
09:56  main: build + unit tests ....................... passed
10:01  build 2.14.0+512 deployed to staging
10:09  end-to-end checkout journeys on staging ........ passed
10:09  release candidate 2.14.0+512 is ready

Try it: API Health Checker Try it: API Response Comparator

Frequently asked questions

References

• Continuous Integration (Martin Fowler)martinfowler.com · 2024The widely referenced description of continuous integration: integrate often, keep the build self-testing and fast, and fix a broken build immediately.
• ISTQB Certified Tester Foundation Level (CTFL) v4.0ISTQB · 2023The foundation syllabus of software testing: test levels and types, test design techniques, test management and tools.
• Eradicating Non-Determinism in Tests (Martin Fowler)martinfowler.com · 2011Why flaky tests destroy a suite's value, their common causes (isolation, asynchrony, remote services, time, resource leaks) and quarantine as a response.