Integrating quality assurance with agile development means spreading the checking through every sprint, so that each piece of work is tested as it is built and nothing counts as finished until it has been. The alternative, which many teams slide into, is to write code for most of the sprint and test in the last two days, or in the following sprint. That is the old test-at-the-end approach repeated every fortnight, with the same weakness: faults are found when there is least time to deal with them.
A word on terms. Quality assurance is everything a team does to prevent faults, including clear requirements, review and automated checks. Testing is the part that looks for faults in what has been built. This article is about the first, and about where each activity sits in the sprint. Who should carry out the testing is a separate question, covered in our article on whether an agile team needs dedicated testers.
What agile itself says about quality
Less than people expect. The Scrum Guide, in its current November 2020 edition, sets sprints of one month or less and lists among the rules of a sprint that “Quality does not decrease”. It defines a Definition of Done as “a formal description of the state of the Increment when it meets the quality measures required for the product”, and says that work which does not meet it cannot be released or even shown at the sprint review. It does not say how anything should be tested.
So the definition of done is the place where quality assurance is built into an agile process, and each team has to write its own. A workable one for a business system might read:
- the change has been reviewed by a second developer;
- automated tests have been written and pass;
- someone other than the author has checked it against the acceptance criteria;
- it works in the test environment with realistic data;
- the person who asked for it has seen it.
The list matters less than the rule that goes with it: an item that does not meet every line is not done, however close the deadline. The rest of this article follows the checks through a sprint, in this order:
| When | What happens | Who is involved |
|---|---|---|
| Before the sprint | Acceptance criteria agreed, with worked examples | Product owner, developer, tester |
| First days | Tests designed, automated tests started alongside the code | Developers, tester |
| Throughout | Each item reviewed and tested as it is completed, automated checks run on every change | The whole team |
| End of the sprint | Users try the finished items, regression checks run | Named users, tester |
| After release | Faults that escaped are traced to the step that should have caught them | The whole team |
Before the sprint: make the work testable
Quality starts with the wording of the request. “Improve the order screen” cannot be tested. “A user can add a second delivery address to an order, and the delivery note prints both” can.
Before an item is accepted into a sprint, it should have acceptance criteria with examples, and whoever will test it should have been in the conversation. Their job at this stage is to ask the awkward questions: what happens if the address is abroad, or the order has already been dispatched. A misunderstanding caught here costs a few minutes. The reasons it costs far more later are set out in our article on why you should start testing early.
An item nobody can describe a test for is not ready to be built.
During the sprint: test as you go
Keep items small. If each item takes two or three days, testing can begin in the first week. A developer building a form to capture customer details can get the name and address fields working and hand them over while the remaining fields are still being built.
Finish one thing before starting the next. A sprint with ten items all nearly complete on the final day leaves no room for testing. Limiting the number of items in progress keeps a steady flow of finished work reaching whoever tests it.
Run automated checks on every change. Each time a developer submits work, the code is built and the automated tests run. This is continuous integration, and it means a fault is reported within minutes to the person who introduced it.
Review every change. A second developer reads the code before it is accepted. Our article on the importance of code reviews explains what that catches.
Fix faults in the sprint that finds them. A fault found in an unfinished item is part of finishing the item. It should not be logged and carried forward.
When testing will not fit, the tempting responses are to lengthen the sprint or push testing into the next one. Both hide the problem. Look for the cause. Usually it is one of these: items that are too large, a test environment that takes too long to prepare, or a manual regression check that has grown until it takes days.
At the end, and from one sprint to the next
The sprint review is a quality check as well as a demonstration. The people who will use the software should try it themselves, on realistic data. For each workflow that matters, one named person in the business should be the one who confirms it behaves correctly.
The second concern is everything built in earlier sprints. Each sprint adds features, so the amount that must be rechecked keeps growing. Rechecking it is called regression testing. Done by hand, it eventually swallows the sprint. Automating the core of it early is what lets an agile team keep its pace after the first few months.
Finally, when a fault does reach users, the team should ask which step ought to have caught it and change that step. A new test is added so the same fault cannot return unnoticed.
Starting with a system that has no tests
Agile development assumes software can be changed safely in small steps. That holds for new software built with tests from the first day. It does not hold for an older system with no automated tests and no record of what it should do, which is the usual state of the systems we are asked to take over.
There, the first sprints deliver a safety net before they deliver features:
- Prove the application can be built from its source code, released in a controlled way and restored from a backup.
- List the workflows the business depends on, and name the person who can confirm each one.
- Record what the system does today in the areas where a mistake would cost most: calculations, permissions and connections to other systems. These recordings are called characterisation tests.
- Rehearse every change on a copy before it goes near the live system.
After that, one line is added to the definition of done: any code that is touched gains an automated check. Coverage then grows where the work is, sprint by sprint, without a separate testing project.
Signs that it is working
- Items are finished steadily through the sprint, not all on the last day.
- Faults found during a sprint are fixed in that sprint.
- Your named users see working software every sprint.
- A fault reported after release leads to a new test.
- Nobody is asked to waive the definition of done to hit a date.
To find out where your own project stands, ask the team for its definition of done. If it is written down and mentions review, automated tests and acceptance by a user, quality assurance is part of the process. If nobody can produce one, writing it is the first thing to do, and it takes a single meeting.