Configuration management tools are not strictly testing tools either, but good configuration management is critical for controlled testing.
To understand this better let us take an example, a test group started testing the software, expecting to find the usual quite high number of problems. But to their surprise, the software seemed to be much better than usual this time – very few defects were found.
Before releasing the software they just want to make an additional check to see if they had the right version. But they discovered that they were actually testing the version from two months earlier (which had already been debugged) with the tests for that earlier version. It was nice to know that this was still OK, but they weren’t actually testing what they were supposed to be or what they should have been testing.
It is really very important to know exactly what it is that we are supposed to test, such as the exact version of all of the things that belong in a system. It is possible to perform configuration management activities without using the tools, but the tools make it a lot easier, especially in complex environments.
Testware needs to be under configuration management and the same tool may be able to be used for testware as well as for software items. Testware also has different versions and is changed sooner or later. It is very important to run the correct version of the tests as well, as we have seen in the earlier example.
Features or characteristics of configuration management tools are:
- To store information about versions and builds of the software and testware.
- Traceability between software and testware and different versions or variants.
- To keep track of which versions belong with which configurations (e.g. operating systems, libraries, browsers).
- To build and release management.
- Baselining (e.g. all the configuration items that make up a specific release).
- Access control (checking in and out).