Testing and quality

This topic is for the person responsible for a working business system that was built by someone else, has no automated tests and still needs to change. The problem it covers is how to make those changes without breaking what already works, and how to judge whether a supplier’s testing is real or just a line in a proposal.

Where to start

• What is regression testing? explains the check that matters most on an existing system, that what used to work still works, and how to start when there are no tests at all.
• An overview of software testing strategies sets out the levels of testing, what each one catches and how to combine them for the system you have.
• Integrating quality assurance with agile development covers what to check before, during and after each round of changes, including on a system that starts with nothing.
• The importance of code reviews explains why a second developer should read every change before it goes live, and what to ask your supplier about theirs.

What people get wrong

The first mistake is to believe one of two opposite things: that a system with no tests cannot be changed safely, or that because it has run for years it is safe to change. Neither is true. It can be changed, but each change needs a check of the things the business depends on, and the first job is writing down what those things are. An order must still total correctly and the overnight import must still run. That list is the beginning of a test suite before any test code is written.

The second is asking for full test coverage as a project in its own right. Writing tests for every part of an old system is expensive, and most of that work is never repaid. Tests pay for themselves where change happens, so they go in first around the areas being changed and the paths that would hurt most if they broke. The suite grows with the work rather than ahead of it.

The third is leaving testing to the end. A fault is cheapest to fix at the moment it is made, and the article on testing early shows what that means in practice. On an existing system it also means testing the release itself. A change that works on the developer’s machine and fails on the live server is a testing failure, not bad luck, and it is common on a system that nobody has released in a while.

How this connects to our work

When CodeFirst takes over a system that someone else built, the assessment starts by proving it can be built from its source code, released in a controlled way and restored from a backup, because none of the checks above mean much until a release can be repeated. Regression checks are then built up around each change rather than in one big job. If you want to know where a system stands before committing to anything, the fixed-price code audit gives you a written report in plain English and a walk-through call, and the fee is credited against any follow-on work. The same habits continue under a software maintenance plan.