13.2. Processes and tools for testing software components

The relative simplicity of the PM allows you to analyze in detail their internal structure and any route of the program execution. This makes it possible to implement two testing strategies: from the structure and from the data. Two methods of program testing correspond to these two strategies: a method for analyzing control flows and analyzing data flows. The methods complement each other, and everyone can be dominant in the initial stages of debugging, depending on the type of object and the conditions of testing.

Strategy 1 Strategy 2

I Building the structure of the PM

I in the text of the program

Preparation of the test to verify the function and data flow in the PM

*

I Isolation and streamlining of routes I by selected criteria

Testing of PM on the formed test

*

I The formation of the test conditions

I in predicates of the selected route

i

Identification of errors in data and calculations in the PM

I Test route PM

I on the formed test

*

Summary of tested PM routes

I Comparison of the implemented route I execution PM and the original text

i

I Identifying and fixing errors

Formation of the structure of the PM on the tested routes

I route inconsistencies

Identification of errors in the calculations on the selected route

one_

Evaluation of the coverage of the initial structure of the PM by the tested routes

\

Assessment of the sufficiency of the performed testing of PM

I

End-testing and registration of the degree of correctness of the PM

Fig.13.3

In the first strategy (Fig. 13.3), the basis is the structure of the PM, built on the text of the program as a graph. In the program column, according to some criteria, the program execution routes and predicate conditions under which they can be realized are selected and ordered. These conditions are used for the preparation of test kits, each of which must be realized along the route taken as a standard in preparing the test. The deviation of the test execution from the originally selected route is considered as an error, the cause of which may be either in the primary structure of the PM or in the implementation of a specific route with this test at the entrance.

After eliminating errors and inconsistencies of the selected and implemented routes for each of them, the data processing process is checked and errors in the results of their conversion are detected. Then, the sufficiency of the performed testing is evaluated according to the degree of coverage of the initial graph of the program using the tested routes, which were distinguished according to the selected or specified criterion. Testing ends with the required coverage of the graph of the program with tested routes or when using the resources allocated for testing. In the latter case, an assessment of the achieved program correctness and registration of this value are necessary.

With this strategy, some distinguished routes may turn out to be fundamentally unrealizable due to the conflicting conditions in successive operators - the vertices of the program graph. As a result, tests may be prepared for some modules that are redundant and do not reflect the actual functioning of the program. However, this strategy has advantages when testing logic programs with a small amount of computation. At the same time, the completeness of testing is rather effectively controlled under various criteria for allocating routes and methods for streamlining them.

In the second strategy (see Fig. 13.3), the requirements of specifications, specific test and reference values, which are prepared by specialists by analyzing variables and predicates in the program text, are taken as a basis. With each test, the program is executed along a specific route that is being registered. In this case, the route and data stream implemented in accordance with the requirements being analyzed are considered as reference components of the program structure.

we. As testing progresses, verified operators are noted and the completeness of test coverage of specification requirements on testing routes is assessed . The accumulation and synthesis of implemented routes allows you to reproduce the structure of the program and control the degree of coverage of each component. To this end, at each stage of testing, the program components are highlighted, which remain untested, for which additional tests must be prepared. However, with such a strategy, it is difficult to assess the degree of coverage and verification of all routes of program execution without using a structural graph constructed from its original text.

This strategy has advantages with a relatively simple structure of the program and with the predominance of computing operators in it. On each route, the orderly variation of variables focuses on the computational part of the program, which is essential for this class of PM. However, the risk of missing combinations of predicates that define an untested route increases. Therefore, it is almost always advisable to use two strategies together.with an emphasis on one of them, depending on the characteristics of the tested PM or its parts. Programs with a predominance of a complex logical structure and with a relatively small computational part, it is advisable to start testing the first strategy and only use data flow analysis (the second strategy) for routes with computational operators. In modules that have a simple structure and contain a significant amount of computations after primary debugging using the second strategy, it is advisable to check the completeness of the structure testing and the completion of debugging using the first strategy.

It is known that in large PSs exhaustive testing, which guarantees the complete absence of defects and errors in them, is fundamentally impossible and the number of defects found in programs even by independent testers has limits. The above-shown tracking of component compliance when verifying the correctness from top to bottom of the initial requirements for the program package and testing the component coverage for compliance with the requirements at each level of their detail may require significant computational, labor and time resources. The actual limitations of the available resources determine the number of unrepaired defects and the attainable correctness.

components and PS as a whole. In practice, it is rather time-consuming and often desirable to have an easier way to regulate and evaluate the quality of testing. There is a problem, how to practically take into account resource constraints and when to stop verification and testing, as well as how correctness of the programs will be achieved and whether it can satisfy the customer and user during the operation of the software .

In the development of complex PS, verification and testing require significant resources throughout the entire life cycle of an SS, and the most critical resource is the allowable time for step-by-step execution of these procedures. The intensity of detection of defects when testing programs almost exponentially decreases depending on the time spent on their detection, and accordingly the intervals between the next manifestations of defects increase. On the basis of experimental data , mathematical models have been created that allow one to predict the time intervals between successive detection of defects or errors in software components with a certain method and stage of verification or testing (see Lecture 10). For example, if testing a specific module or component of a PS reveals a certain number of defects (for example, 5 or 10), then the models allow estimating the resources (time) required to detect the next defect and the profitability of continuing the testing with the method used. A rational change in the strategy or method of detecting defects can lead to a short-term increase in the intensity of their manifestation, and then to a gradual decrease in it.

For using such models in a specific firm for a certain class of projects, with a certain qualification of developers, the number of defects that can be detected by testers at a certain stage of testing a software component over the allotted time experimentally can be established. If during this time the number of defects is detected less than a priori set, then this may mean either a very good job of programmers-developers, or an insufficient quality of tests and their performers. Some additional testing may help solve this alternative. Regular registration of the number of detected defects for a specified time by certain testers, in the results of the work of specific programmers,

It will allow to estimate the average intensity of detection of defects during testing at the enterprise when developing certain types of projects. Such estimates can be used to predict the quality achieved of the corresponding program components. In this case, it is advisable to take into account the categories of detected defects according to the degree of influence on the results of the program's operation: critical - unacceptably distorting results, dangerous - threatening with small distortions of results in rare cases, as well as little or no effect on the results.

Modern systems of systematic testing and debugging of software components of high and controlled quality should provide:

• convenient, friendly dialogue in the symbolic and graphical types of users with automation tools and, mainly, paperless program testing;

• the use of a sufficiently developed and effective design database (repository) for accumulating and storing various information about the developed programs, their versions, test plans, test and reference data, corrections made;

• automatic detection by static methods of typical errors in the source texts of programs caused by violations of the formalized rules of programming semantics, the structural construction of modules and the use of data;

• automated testing planning, issuing recommendations to users on the systematic application of methods, strategies and tools for dynamic debugging;

- effective implementation of debugging tasks in order to achieve maximum program correctness in the context of limited testing resources;

• evaluation of the achieved program correctness according to the selected testing criteria and determination of the main indicators of the quality of the created program components;

• automated registration and documentation of all changes performed in programs and accounting for versions of program modules and groups of programs in which adjustments are made.

To debug software components that are part of complex software and are suitable for reuse in various projects, a set of automation tools is needed, using the basic modern methods of detecting errors and defects in programs. These funds can be divided into (Fig. 13.4):

• static - analyzing specifications and source codes of programs without their execution in the object code;

• dynamic, using which programs function in object code and are suitable for their actual use.

These tools are combined by means of an interactive dialogue with users and the design database. The static debugging group includes tools that automate the structural analysis and verification of the formal correctness of program texts and the identification of appropriate types of errors. In addition, they should provide automated data retrieval that supports the detection of errors during the subsequent use of dynamic tools. Means of preparing data for planning testing should ensure the allocation of typical structures in the module (cycles, alternatives, execution routes) and their ordering according to certain rules. Allocation of the routes of program execution and the conditions of their formation allows to determine the boundaries of the regions of change of variables that affect the formation of test sets and their volume. As a result of static analysis, programs can automatically create ordered sets of its characteristics, which are used to prepare test suites, as well as to control the completeness of testing modules performed during dynamic debugging.

The group of static debugging tools also includes the means of calculating the duration of the execution of modules and components. These tools make it possible to obtain approximate values ​​and distributions of the program’s account duration analytically, without its execution on a computer. As a result of the calculations, program components are identified that require an excessively long counting time on the implementing computer, and data are prepared for a general check of the realizability of the software in real time,

A group of dynamic debugging tools can be divided into fixed assets that directly ensure the execution of programs in accordance with debugging tasks, and tools that analyze the testing performed, its results and corrections made. The main funds include:

• translation of programs, tests and tasks from the debugging language;

• execution of the program for the debugging task in accordance with the plan and test scenario;

• data recording of test results.

The means of translation of tasks from the debugging language provide processing and preparation for the execution of tests and the script of the debugging task. The task specifies the program being tested, monitored and recorded variables and program states in the process of execution. Test values ​​are converted to a form suitable for execution by the program being debugged. Operators of the debugging task are combined with the program being debugged or prepared for execution in emulation or interpretation mode.

The means of executing the program for the debugging task control the implementation of the statement operators in the process of executing the program being debugged. In the process of test processing, a preliminary selection of test results is performed in accordance with the instructions of the debugging task operators. In some cases, the results obtained can be automated compared with the reference values ​​to detect errors.

The means of recording and summarizing data on test results converts the recorded data of the functioning of the program being debugged into a debugging language for interactive user interaction. To reduce redundant information, editing and selection of the results of the execution of debugging operators are performed. The display of test results in mnemonic and graphical forms can be provided by unified means of interactive user interaction with computers.

For each debugged software component, storage of the main tests and debugging tasks with which the testing was conducted should be provided. This allows you to maintain high quality components and, if necessary, make adjustments to them. When making such changes, the composition of the tests and tasks used is expanded and modified accordingly. Together with the tests it is advisable to store the results of the evaluation of the completeness of testing and the achieved correctness of the software component. These data, together with an assessment of the complexity of the component, structural and informational characteristics can be combined into a passport of the certification of the component.

To support configuration management of software components, it may be useful to store at a certain time interval.

changes carried out and errors identified (see Lecture 16). This data can be used to predict the debugging processes of programs and the timing of achieving a given quality. They allow you to identify the statistical characteristics of errors and skillfully manage the process of modifying programs.

Testing planning is a creative process, and automation tools are mainly designed to prepare the raw data used in planning. This information can be divided into three groups of data:

• about cycles in the program;

• about the routes of the program;

• on predicates that determine the program execution routes and the boundaries of variable variation areas.

To prepare this data, the program text in the programming language and the weighted graph model of the program are used. This model can be specially prepared by means of automated test planning or by means of controlling the structure and determining the duration of program execution. The main part of the data for planning testing can be obtained by automatic calculation of the program text using the developer's instructions on the selection criteria and strategy for streamlining routes. A developer’s dialogue with automation tools is advisable to clarify the criteria and strategies for the formation of routes depending on the complexity of testing, as well as to eliminate cycles and acyclic routes that are not implemented by a combination of conditions in predicates. The dialogue can also be useful in preparing a weighted graph model of the program when it is necessary to enter estimates of the branching probability values ​​at the vertices of the graph and the characteristics of loop iterations.

In the program, first of all, the cycles to be tested should be automatically allocated . To do this, use the developer's instructions on the strategy of allocating routes when testing cycles. In addition, you should enter instructions on the number of iterations of cycles and their links to the routes of execution of cycles. As a result, the developer displays data about the routes in cycles that are to be tested according to the chosen strategy

See also

• 13 VERIFICATION, TESTING AND EVALUATING THE CORRECTNESS OF SOFTWARE COMPONENTS