How to manage Regression Risk and evolve manual, automated test cases in Agile methodology?

Agile teams largely make incremental delivery, what that essentially means is, as the number of iteration increases, the size of the product also increases significantly, thus the scope of development and testing also increases. The risk of regression is proportional to the scale of the product. As the product size increases the risk of regression too increases, hence testers may need to have good mitigation plan in place to take care of this risk.

Extensive automation is highly recommended to get rid of regression risk. The automation scripts are kept updated in every iteration. The testers also play role in managing the technical debt of a legacy product. If the technical debt increases along with the size of the product, it becomes unmanageable for a peak point. All the test cases, test scripts, automation tests are checked in into the configuration management tool so that they can be retrieved and compared during the times of need. It should be noted that keeping the history handy, will always help in audit trial during troubleshooting a major problem.

Evolving Manual and automated Test Cases

Testers often do routine job every iteration; however, there might be excessive work load during some iteration it may be difficult to juggle around several issues. The testers at the beginning of the iteration will have to get together to prepare a test strategy to deal with the unknowns and known and proceed accordingly. They testers team should also review the tests written earlier and discuss the probably touch points on where they need to add or delete the scripts.

Doing automation of every test case will provide easy access to the new defects that are coming up during development. The automation test cases document must be constantly updated in order to keep the ball rolling. Teams may also review the functionality that was tested till date and may retrospect on improving the same.

The daily build contains automated regression tests and it connected to the continuous integration environment, as soon as it detects the new check-in, it triggers a new build and same is deployed after the build is completed. If the build fails to due to failure in automation regression test fails, the team needs to stop their work immediately and investigate the reasons for failure. The test could have failed due to various reasons or test case is no longer needed for the system and it may have to be replaced with another suitable test case. When a test case is failed due to inherent defect that exists in the system, it is recommended that the team fixes the defects before moving to new feature development.

Additionally, apart from test automation, the following testing tests may also be automated:

  • Generating test data
  • Test data loading
  • Deployment of builds in test environments
  • Back-out test environment
  • Comparing data outputs