Quality Assurance Metrics

According to international standard ISO 14598:

A metric is a quantitative scale and a method that can be used to measure.

From our side, we add that the introduction and use of metrics is necessary to improve control over the development process, and in particular over the testing process , which we will consider further.

The purpose of the test control is to obtain feedback and visualization of the testing process . Information necessary for control is collected (both manually and automatically) and is used to assess status and make decisions, such as coverage (for example, covering requirements or code with tests) or exit criteria (for example, testing end criteria). Metrics can also be used to assess progress in planned work and budget

Create, use and analyze metrics

In our opinion, for greater clarity, it makes sense to group the metrics by the types of entities involved in quality assurance and software testing, namely:

1. Test Case Metrics (Test Cases)
2. Bug / Defect Metrics
3. Task Metrics

Let's take a closer look at each of them:

Test Case Metrics

Bug metrics

We would like to note that the metrics "Open / Closed Bugs", "Bugs by Severity" and "Bugs by Priority" visualize well the degree to which the product is approaching the achievement of quality criteria by bugs. We have requirements for the number of open bugs, after each iteration of testing, we compare them with real data, thus seeing the places where we need to add in order to achieve the goal as quickly as possible.

The "Reopened / Closed Bugs" and "Rejected / Opened Bugs" metrics are aimed at tracking the work of individual members of the development and testing groups.

Example one :
Suppose we have a situation where the number of reopened bugs after a repair does not decrease or even grow. This is a signal that it is necessary to analyze the causes, since a similar situation may show that:

1. Function requirements can be interpreted in different ways.
2. The tester did not accurately describe the problem.
3. Poor surface solution (fix bug)

The second example will show why the "Rejected / Opened Bugs" metric is needed:
We observe that the percentage of rejected bugs is very large. This may mean:

1. Function requirements can be interpreted in different ways.
2. The tester did not accurately describe the problem.
3. The developer does not wish to correct the mistake made by him or does not consider that this is in fact a mistake. (This problem is a direct consequence of the 2nd, which arose due to an inaccurate description)

All these problems significantly destabilize the situation on the project. Therefore, when they occur, it is recommended to hold a short conversation with the leaders of the project teams in order to subsequently reduce the number of reopened and rejected defects.

Task Metrics

Task metrics can be different, we have resulted only two of them. Also interesting may be the metric for the execution time of tasks and many others.

* * *

In conclusion, we would like to note that the availability of the necessary metrics and graphs reflecting the change in the state of the project over time will allow you to improve not only the testing process, but also the development as a whole, and also facilitate the procedure for analyzing the completed project, which will allow you to avoid past mistakes.