What is a proof-of-concept or piloting phase for tool evaluation in software testing?

One of the ways to do a proof-of-concept is to have a pilot project as the first thing done with a new tool. This will use the tool on a small scale, with sufficient time to explore different ways in which it can be used. Objectives should be set for the pilot in order to accomplish what is needed within the current organizational context.

A pilot tool project expected to have issues or problems – they should be solved in ways that can be used by everyone later on. The pilot project should experiment with different ways of using the tool. For example, different settings for a static analysis tool, different reports from a test management tool, different scripting and comparison techniques for a test execution tool or different load profiles for a performance-testing tool.

The objectives for a pilot project for a new tool are:

  • To learn more about the tool and in detail.
  • To see how the tool would fit with existing processes or documentation, how those would need to change to work well with the tool and how to use the tool to streamline existing processes;
  • To decide on standard ways of using the tool that will work for all potential users (e.g. naming conventions, creation of libraries, defining modularity, where different elements will be stored, how they and the tool itself will be maintained);
  • To evaluate the pilot project against its objectives (have the benefits been achieved at reasonable cost?).