Software Testing Completion Criteria: Importance and Examples

The criteria for completing software testing are defined in order to make sure that the functionality and quality of the product being developed have been verified to a sufficient degree.

The main criteria for completing testing:

Testing must be continued, i.e. it is assumed that the more errors are found during testing, the more we can still find

The number of detected errors tends to zero; therefore, the testing process can be completed.

For example, 90% code coverage by tests is required, which means that most of the code has been tested and checked for errors.

4. Quantitative reliability indicators
The criterion for completing testing by reliability indicators, which are calculated from the reliability module of the software product.

Other common criteria for completing testing:

1. Test coverage: The software code must be adequately covered by tests in order to verify all the main functions and components. This includes unit testing of individual modules, integration testing of the interaction between modules, and system testing of the whole product as a whole.

2. Successful execution of test scenarios: All planned test scenarios must be executed without errors and problems. Testing should cover various usage scenarios, as well as positive and negative test cases.

3. Error handling: Errors found during testing must be handled adequately. The development team must investigate and fix the errors found, and then retest the fixed components to make sure they work correctly.

4. Time and resources: Testing must be completed within a given time period and with the available resources. Planning and managing testing time are important aspects of achieving completeness.

5. Quality assurance: The evaluation of test results must reach a level at which the development team can confidently state the quality of the software and its readiness for release.

6. Customer acceptance: The completeness of testing can be confirmed and accepted by the customer or stakeholders. This may include formal approval and acknowledgment of the testing performed.

Test completion criteria – examples:

• The specified coverage has been achieved.

• There are no showstoppers (blockers) or critical bugs.

• There are very few known bugs of medium or low priority that do not affect the use of the product.

. There is a defined process and criteria that determine whether we can move from one testing level to another. It is captured as part of the test closure report for that test level. Several examples of combinations of completion criteria for different testing levels can be defined as follows:

• Component testing: 100% statement coverage, 95% decision coverage, no known bugs - test closure in component testing usually has a limit on achieving unit test coverage. It also guarantees that there are no critical defects that could affect component integration testing. The test strategy defines the percentage of unit test coverage, which is usually maintained at more than 80%.
• Integration testing (for both components and systems): 90% parameter coverage, 60% interface coverage, no known bugs - test closure at this level involves integrated components whose testing is completed (for example, a cart with address validation, validation with a payment gateway, etc.). If the testing of all integrated components is complete and there are no critical defects, testing moves on to the next testing level. This level is called system testing.
• System testing: 90% requirements coverage, 100% equivalence class coverage for specific requirements, no known failures of criticality P1 or P2, a stable number of failures per hour of testing over more than 20 hours of testing - since system testing is the last testing level before we hand the product to customers for user acceptance, the closure report is detailed. Usually the test strategy defines KPIs (key performance indicators) that must be met before we can successfully complete system testing. An example of KPIs is as follows.
  • 100% system test execution
  • 95% pass rate
  • 0 P1/P2 defects and fewer than 50 P3/P4 defects (Defect priority: P1 High; P2 Medium; P3 Low.)
  • Cross-browser/cross-device testing completed with a pass rate of >90%.
  • Accessibility verification completed
  • Analytics testing completed
  • Performance testing completed with acceptable and agreed issues
  • Security testing completed, no serious defects expected.

As you can see, there are several KPIs that need to be tracked in order to complete system testing. Thus, the test completion report is quite exhaustive. This report is presented to business stakeholders, and based on the results they decide whether user acceptance testing can begin.

• User acceptance testing:100% business procedure coverage, no known failures of criticality P1 - this is the last testing level before the software is launched into production. The test completion report usually contains the execution status and open defects. This report determines whether the software release can go into the production environment.
• Completion of a debugging release: For debugging releases we usually do not perform comprehensive testing, and the test completion report may list the new features that were added, for which appropriate testing is carried out, together with open defects.
• Cancellation of a test project: in rare cases a test project may be canceled or postponed, usually because the software is no longer needed or because of a strategic business decision. In such cases the closure report states the testing completed so far, the defects, and open dependencies. The completion report helps ensure that when the project is restarted we do not have to start from scratch.

Test completion criteria – significance:

The significance of the criteria for completing software testing is as follows:

1. Quality assessment: The completion criteria help assess how fully and effectively testing has been carried out. They make it possible to determine whether all the necessary test scenarios have been executed, the code coverage by tests, and other important aspects. This makes it possible to be sure of the quality of the software under test and its readiness for use.

2. Risk management: The criteria for completing testing help manage the risks associated with insufficient or incomplete testing. They make it possible to make sure that critical defects have been found and fixed, that the functionality of the software has been verified, and that the system is ready for real operation. This reduces the risk of problems and of unmet user needs.

3. Meeting requirements: The criteria for completing testing make it possible to make sure that all the requirements of the customer or business users have been taken into account and tested. They serve as a basis for verifying the software's compliance with the stated requirements, functionality, and user expectations. This is important for achieving the project's goals and satisfying the customer's needs.

4. Resource efficiency: The criteria for completing testing help optimize the use of resources such as time, budget, and effort. They make it possible to set clear boundaries and limits for testing in order to reach the required level of quality without unnecessary costs.

5. Trust and confidence: The criteria for completing testing help build trust and confidence in the quality of the software. They serve as a basis for confirming that the necessary checks have been performed and that the system is ready for successful operation. This is important for customers, users, and other stakeholders who rely on the functionality and reliability of the software.

• If the exit criterion is not met, testing cannot be stopped.
• It is necessary to change the criterion for stopping testing or to increase the testing time, depending on the quality of the product.
• Any changes to the test completion criteria must be documented and signed off by stakeholders.
• The software under test can be released after the exit criteria have been successfully met.

All these aspects underline the significance of the criteria for completing testing, which help ensure quality, manage risks, meet requirements, and build trust in the software. Taken together, all the criteria help determine the completeness of software testing and the product's readiness for the next stage, such as release or deployment. However, it should be noted that the completion criteria may vary depending on the specific requirements of the project and the quality standards established in the organization.

See also

• [[b5187]]
• [[b7769]]
• [[b5192]]
• [[b5191]]

See also

• Types and kinds of software testing. Testing levels and testing methods
• Methods to ensure the high quality of the developed software
• Mobile testing: features and stages. Mobile application architecture
• Web testing: features, kinds and sequence