Blog

Levels of testing: unit, integration, system and acceptance

Each level of testing checks a bigger piece of the software than the last. Here is what unit, integration, system and acceptance tests catch, on one example.

Software is tested at more than one scale. The smallest checks look at a single function with everything around it held still. The largest look at the finished product through the eyes of the people who will use it. Each scale in between has its own job, and the scales are called levels of testing. Knowing them helps when you read a project plan, when you sign off a build, and when you try to work out why a fault got as far as production.

Most guides name four levels: unit, integration, system and acceptance. The ISTQB Certified Tester Foundation Level syllabus (v4.0.1, section 2.2.1) describes five, because it splits integration into two: the connections between the parts of one system, and the connections between that system and outside services. This post follows the four common names and points out where the fifth fits.

What the levels are

A test level is a group of tests aimed at the software at one stage of assembly, from a single component up to the whole system. What separates one level from the next is what is under test, what the tests are trying to show, and which faults they are best placed to find.

To keep the examples concrete, every section below tests the same feature: a discount code at an online store’s checkout. A shopper types a code, the basket total drops, and the order goes through at the lower price.

Four levels of testing applied to one discount code feature, each tested piece larger than the last. Unit testing: one function on its own; is the discounted total right? Integration testing: parts that call each other; is the right price saved on the order? System testing: the whole store, end to end; can a shopper apply a code and pay? Acceptance testing: the business need; do the rules match the planned promotion?

Unit testing

A unit test checks one small piece of code on its own: a function, a class, a single component. Anything that piece depends on, such as a database or another service, is usually replaced with a stand-in that returns fixed answers, so a failure points at the code under test rather than at something around it. ISTQB calls this level component testing and notes that developers normally run these tests in their own development environment.

In the example: the function that works out the new total. Given a basket of $80 and a 10 percent code, does it return $72? What does it return for a code that has expired, for a basket below the minimum spend, or for a fixed-amount code larger than the basket itself?

Unit tests are fast and narrow, so a project can afford to run the whole set on every change. What they cannot show is whether the pieces work once they are joined.

Integration testing

Integration testing joins parts that have each passed their own tests and checks the places where they meet. Faults here come from mismatched assumptions: one part sends a price in cents and the other reads it as dollars, or a field is optional on one side and required on the other.

In the example: the checkout page sends the code to the discount logic, gets a total back and writes the order to the database. Does the order record the discounted price, the code used and the amount taken off? If the discount logic refuses the code, does the checkout show the shopper a clear message instead of failing silently?

This is the level ISTQB splits. Component integration testing covers connections inside the system, like the ones above. System integration testing covers connections to things outside it, such as the payment provider or the service that sends order emails. This level usually needs a test environment set up to behave as closely as possible like the live one, including test versions of those outside services where they offer one.

System testing

System testing treats the software as one finished product and checks it end to end, against what the specification says it should do. It covers behavior (does each task work from start to finish?) and qualities that only show up in the whole, such as speed, security and how easy the product is to use. ISTQB notes that this level may be run by a test team separate from the developers who wrote the code.

In the example: in a test copy of the store, a tester adds items to the basket, applies the code, pays with a test card and receives the confirmation email showing the lower price. The same journey is tried on a phone and on a slow connection, and with a code typed in lower case, with a stray space, or entered twice.

Acceptance testing

Acceptance testing asks a different question from the levels below it. They check that the software does what the specification says; acceptance checks that it does what the business actually needs, and that it is ready to go live. Ideally, the people who will use it run these tests. ISTQB lists several forms: user acceptance testing, operational acceptance testing, contractual and regulatory acceptance testing, and alpha and beta testing. In plain terms, operational acceptance asks whether the system can be run, backed up and restored, and alpha and beta testing put it in front of real users before a full release.

In the example: the store’s marketing lead tries the codes for next month’s promotion. Each code should stop working at midnight on its end date, and none should combine with the sale prices already in the catalog. If the specification never mentioned the sale prices, every lower level can pass and acceptance will still fail. Catching that gap before launch is the point of this level.

Acceptance criteria are not only about features. A performance target can be one of them. On our custom web development projects, every page has to meet its performance budget in lab tests, and meeting it is a condition of launch.

Regression testing at every level

Regression testing is not a fifth level. It runs at all of them. Every change, whether a new feature or a bug fix, can break something that used to work, sometimes in a part of the system nobody touched. A regression run repeats the existing tests after a change to show that nothing else has moved. A related check, confirmation testing, reruns the test that exposed a fault to show that the fix really fixed it.

Because the same regression tests are run again and again, they are the first tests worth automating. This matters long after launch, too. Software that is live keeps changing: libraries get security updates, platforms release new versions, a payment provider retires an old API. Keeping up with changes like these is adaptive work, one of the four types of software maintenance. Each of those changes is a reason for a regression run, which is where regression testing earns its keep. On a software maintenance retainer, security patches and dependency updates are applied and tested before release.

How the levels fit together

The levels build on one another, but they do not have to happen one after another. In a sequential plan, each level often has to finish before the next begins, and the V-model draws that by pairing each design stage with the test level that checks it. In iterative work, all of the levels can run in every iteration, and they overlap in time. Our guide to SDLC models covers how to choose between those approaches.

A common way to picture the balance is the test pyramid. Many small, fast, isolated unit tests sit at the base; fewer integration tests sit above them; a small number of slow end-to-end tests sit at the top. The shape is a guide to where effort goes, not a rule. A product that is mostly connections to other services may need more integration tests than the triangle suggests.

How much testing a build needs at each level depends on what a fault would cost. A first version built to learn whether anyone wants a product, the usual aim of MVP development, carries less risk than a payment system used by thousands of people. Even so, the paths that handle money or personal data deserve tests at every level from the start.

What to ask a development team about testing

If you are commissioning software, you do not need to read the tests. A few plain questions will tell you how a team works:

  • Which levels do you test at, and who runs each one? A clear answer names the levels and the people. A vague one (“we test everything”) tells you less.
  • What runs automatically on every change? Unit and regression tests usually can. Ask what happens when one fails.
  • How will we take part in acceptance testing? Acceptance needs your people and your knowledge of the business, so it needs time in your calendar before launch.
  • What are the acceptance criteria, and are they written down? A criterion you can check, such as a page that loads within a set time, is better than “works well”.
  • What is tested after launch, when something changes? The answer should cover updates and fixes, not only new features.

Questions people ask

What are the four levels of testing?

Unit testing, integration testing, system testing and acceptance testing, run roughly in that order as the software is assembled. Each one tests a larger piece than the one before. ISTQB counts five, because it treats integration inside the system and integration with outside services as separate levels.

Is regression testing a level of testing?

No. A level describes how much of the software is under test. Regression testing describes why a test is being run: to check that a change has not broken anything. It can happen at any level.

What is the difference between system testing and acceptance testing?

System testing checks the finished software against its specification. Acceptance testing checks it against the business need, usually with the people who will use it. A build can pass system testing and fail acceptance when the specification missed something the business needed.

The full service behind this post: Custom Web Development Services.

Ready to scope your project?

Send us two sentences about what you need. We reply within one business day with the next step and any questions we need answered first.

Get a Quote