Lecture
Debugging and testing programs.
1. Debugging the program
Debugging, as we have said, is of two types:
Syntax debugging. Syntax errors are detected by the compiler, so correcting them is easy enough.
Semantic (semantic) debugging. Its time comes when there are no syntax errors, but the program gives incorrect results. Here, the compiler itself will not be able to reveal anything, although in the programming environment there are usually debugging aids that we will talk about later.
Debugging is the process of localizing and fixing errors in a program.
No matter how carefully we write, debugging almost always takes longer than programming.
2. Localization of errors
Localization is the location of the error in the program.
In the process of searching for errors, we usually perform the same actions:
run the program and get the results;
we compare the results with the reference and analyze the inconsistency;
detect the presence of an error, hypothesize its nature and place in the program;
check the text of the program, correct the error, if we found it correctly.
Ways to detect errors:
Analytical - having a sufficient understanding of the structure of the program, we review its text manually, without a run.
Experimental - we run the program using debug printing and tracing tools, and we analyze the results of its work.
Both methods are convenient in their own way and are usually used together.
3. Debugging principles
Principles of error localization:
Most of the errors are detected at all without launching the program — just by carefully looking through the text.
If debugging is at a standstill and fails to detect the error, it is better to postpone the program. When the eye is "zamylen", work efficiency stubbornly tends to zero.
Extremely handy tools are development environment debugging mechanisms: tracing, intermediate value control. You can even use a memory dump, but such radical actions are extremely rarely needed.
Experimentation like "what will happen if you change plus to minus" - you need to avoid all the forces. Usually this does not produce results, but only confuses the debugging process, and also adds new errors.
The principles of error correction are even more similar to the laws of Murphy:
Where one error is found, there may be others.
The probability that an error is found correctly is never equal to one hundred percent.
Our task is to find the error itself, not its symptom.
I want to clarify this statement. If the program persistently returns a result of 0.1 instead of a reference zero, it is not a simple rounding off. If the result turns out to be negative instead of the reference positive, it is useless to take it modulo - we get instead of solving the problem of nonsense with fitting.
Correcting one mistake, it is very easy to add a couple more to the program. The "induced" errors are a real scourge of debugging.
Error correction often forces us to return to the program design stage. This is unpleasant, but sometimes inevitable.
4. Debugging methods
Power methods
- Use dump (print) memory.
This is interesting from the cognitive point of view: you can thoroughly understand the machine processes. Sometimes this approach is even necessary - for example, when it comes to allocating and freeing memory for dynamic variables using the undocumented features of the language. However, in most cases we get a huge amount of low-level information, to deal with which - you do not want the enemy, and the search results are vanishingly low.
- The use of debug printing in the text of the program - arbitrarily and in large quantities.
Getting information on the performance of each operator is also interesting. But here we are again confronted with too much information. In addition, we are awesomely cluttering up the program with additional operators, getting unreadable text, and at the risk of making a dozen new errors.
- Using automatic debugging tools - tracing with tracking of intermediate values of variables.
This is probably the most common way to debug. One need not forget that this is only one of the ways, and it is often unprofitable to use it always and everywhere only.
Difficulties arise when you have to track too large data structures or a huge number of them. It is even more problematic to trace the project, where the execution of each subroutine leads to the call of a couple of dozen others. But for small trace programs it is quite enough.
From the point of view of "correct" programming, force methods are bad because they do not encourage task analysis.
Summarizing the properties of force methods, we get practical tips:
- use tracing and tracking of variable values for small projects, individual subprograms;
- use the debug print in small quantities and in the case;
- leave a memory dump in the most extreme case.
Method of induction - analysis of the program from the particular to the general.
Look through the symptoms of an error and determine the data that have at least some relation to it. Then, using tests, we exclude the unlikely hypotheses, until one remains, which we are trying to clarify and prove.
The deduction method is from general to specific.
We hypothesize, which may explain the error, even if not completely. Then, using tests, this hypothesis is verified and proved.
Reverse motion by the algorithm.
Debugging begins where the wrong result was first encountered. Then the work of the program is traced (mentally or with the help of tests) in the reverse order until the place of a possible error is discovered.
Test method
Let's look at the error localization process with a specific example. Let given a small program that gives the value of the maximum of the three numbers entered by the user.
var
a, b, c: real;
begin
writeln ('The program finds the value of the maximum of the three numbers entered');
write ('Enter the first number'); readln (a);
write ('Enter the second number'); readln (b);
write ('Enter the third number'); readln (c);
if (a> b) and (a> c) then
writeln ('The first number was greatest', a: 8: 2)
else if (b> a) and ( a > c) then
writeln ('The second number was greatest', b: 8: 2)
else
writeln ('The third number was greatest,' b : 8: 2);
end.
Both highlighted errors can be detected with the naked eye: the first one is clearly admitted through inattention, the second due to the fact that the copied string was not corrected.
Test datasets should take into account all solutions, so we choose the following sets of numbers:
Data Expected Result
a = 10; b = -4; c = 1 max = a = 10
a = -2; b = 8; c = 4 max = b = 8
a = 90; b = 0; c = 90.4 max = c = 90.4
As a result of the program, we, however, obtain the following results:
For a = 10; b = -4; c = 1:
The greatest was the first number 10.00
For a = -2; b = 8; c = 4:
The third number turned out to be the largest 8.00 For a = 90; b = 0; c = 90.4:
The largest was the third number 0.00
The conclusion in the second and third cases is clearly wrong. We'll figure out.
1. Tracing and intermediate printing
Add intermediate printing:
- output a, b, c after input (we check if the data was received correctly)
- output the value of each of the conditions (we check if the conditions were written correctly)
The listing of the program has increased significantly and became like this:
var
a, b, c: real;
begin
writeln ('The program finds the value of the maximum of the three numbers entered');
write ('Enter the first number'); readln (a);
writeln ('You entered a number', a: 8: 2); {excellent print}
write ('Enter the second number'); readln (b);
writeln ('You have entered a number', b: 8: 2); {ex. print}
write ('Enter the third number'); readln (c);
writeln ('You have entered a number', c: 8: 2); {ex. print}
writeln ('a> b =', a> b, ', a> c =', a> c, ', (a> b) and (a> c) =', (a> b) and (a> c)); {excellent print}
if (a> b) and (a> c) then
writeln ('The first number was greatest', a: 8: 2)
else begin
writeln ('b> a =', b> a, ', b> c =', b> c, ', (b> a) and (b> c) =', (b> a) and (b> c)); {ex. print}
if (b> a) and (a> c) then
writeln ('The second number was greatest', b: 8: 2)
else
writeln ('The third number was greatest,' b: 8: 2);
end;
end.
In principle, even when typing, we have a good chance to catch an error in the condition: these pieces of code are usually not interrupted, but copied, and if you take the trouble a little while to think about it, the error is easy to find.
But let's assume that the eye is “zamylen” perfectly, and failed to find the error.
The output for the second case is obtained as follows:
The program finds the value of the maximum of the three numbers entered
Enter the first number -2
You entered the number -2.00
Enter the second number 8
You entered the number 8.00
Enter the third number 4
You have entered the number 4.00
a> b = FALSE, a> c = FALSE, (a> b) and (a> c) = FALSE
b> a = TRUE, b> c = TRUE, (b> a) and (b> c) = TRUE
The greatest was the third number 8.00
With the input everything is in order. However, in this doubt, and so it was a bit. But as for the second group of print operators, the picture came out interesting: as a result, the correct number is displayed (8.00), but the wrong word ("third" and not "second").
Probably a problem in the output of results. We carefully check the text and find out that in the latter case it is not b, but b that is displayed. However, this does not apply to solving the current problem: after correcting the error, we get the following result for the numbers -2.0, 8.0, 4.0.
The largest was the third number 4.00
Now the error is localized to the settlement block and, after some efforts, we find and correct it.
2. Method of induction
Judging by the results, an error occurs when the maximum number is the second or the third (if the maximum is the first, then it is determined correctly, you can program another two or three tests to prove it).
View all related to variables b and c. There are no problems with the input, and as for the output, we quickly come across the replacement of b with c. We fix.
As you can see, undetected errors in the program remain. We look through the calculated block: everything that relates to the maximum b (the maximum with is obtained "otherwise"), and we discover the notorious problem "a> c" instead of "b> c". The program is debugged.
3. The deduction method
Incorrect results in our case may be due to an error in:
- data entry;
- settlement block;
- actually output.
For proof, we can use debug printing, tracing, or just a set of tests. In any case, we identify one error in the calculation and one in the output.
4. The reverse movement of the algorithm
Knowing that the error occurs when displaying the results, consider the code, starting with the operators of the output. Immediately find the extra b in the writeln statement.
Next, we look at the specific branch of the conditional operator, where the result came from. For the values of -2.0, 8.0, 4.0, the calculation goes on the branch with the condition if (b> a) and (a> c) then ... where we immediately find the desired error.
5. Testing
In our task, for the most complete set of data, you need to select variables such that
a> b> c
a> c> b
b> a> c
b> c> a
c> a> b
c> b> a
Analyzing the results obtained in each of these cases, we come to the fact that problems arise when b> c> a and c - maximum. Knowing these details, we can focus on specific areas of the program.
Of course, in real work, we don’t paint every step so boringly, we don’t resort exclusively to one method, and in general we don’t often wonder how to look for blunders. Now that we have dealt with all approaches, everyone is free to choose those that seem the most convenient.
5. Debugging Tools
In addition to techniques, it would be good to have an idea about the tools that help us identify errors. It:
1) Emergency printing - output messages about the abnormal completion of individual blocks and the program as a whole.
2) Printing in the program nodes - output intermediate values of parameters in places chosen by the programmer. Usually, these are critical parts of the algorithm (for example, the value on which the further course of execution depends) or components of complex formulas (separately calculate and deduce the numerator and denominator of a large fraction).
3) Direct tracking:
- arithmetic (after what is equal, when and how the selected variables change),
- logical (when and how the selected sequence of statements is executed),
- control over the index out of allowable limits,
- tracking calls to variables,
- tracking calls to routines,
- checking the index values of the elements of arrays, etc.
Current development environments often suggest that we respond to an emerging problem interactively. In this case, you can:
- view the current values of variables, memory status, part of the algorithm where the failure occurred;
- interrupt program execution;
- make changes to the program and re-run it (in compiler environments, this will require recompiling the code, you can continue to run the interpreter directly from the modified statement).
6. Classification of errors
If you are depressed by stupid mistakes in the text of the program - do not be discouraged. Errors are not smart at all, although they can relate to very different parts of the code:
- data access errors,
- data description errors,
- calculation errors,
- errors when comparing
- errors in the transfer of control,
- I / O errors,
- interface errors
and so on to infinity
7. Debugger Tips
1) Check carefully: the error is most likely not in the place in which it seems.
2) Often it turns out to be easier to identify those parts of the program where there are no errors, and then look in the others.
3) Carefully follow the declarations of constants, types and variables, input data.
4) With consistent development, it is necessary to write drivers and stubs especially carefully - they themselves can be a source of errors.
5) Analyze the code, starting with the most simple options. The most common errors are:
- the values of the input arguments are taken in the wrong order,
- the variable is not initialized,
- when the module is repeated, the variable is not reinitialized,
- instead of the intended full copy of the data structure, only the top level is copied (for example, instead of creating a new dynamic variable and assigning it the desired value, the address is stupidly copied from an existing variable),
- brackets in a complex expression are placed incorrectly.
6) With persistent long-term debugging of the eyes "zamylivaetsya." A good trick is to seek help from another person in order not to repeat erroneous reasoning. True, it often remains a problem to convince this other person to help you.
7) Error, most likely will be yours and will be in the text of the program. Much less often it turns out:
in the compiler,
operating system
hardware,
electrical wiring in the building, etc.
But if you are absolutely sure that there are no errors in the program, look at the standard modules to which it refers, find out if the development environment version has changed, eventually, just restart the computer - some problems (especially in DOS-based environments run by under Windows) occur due to incorrect work with memory.
8) Make sure that the source code of the program corresponds to the compiled object code (the text can be changed, and the launched module that you are testing is compiled from the old version).
9) Obsessive search for one mistake is almost always unproductive. It does not work - postpone the task, grab the writing of the next module, at worst, do documentation.
10) Try to spare no time to clarify the cause of the error. This will help you:
fix the program
detect other errors of the same type
do not do them further.
11) If you already know the symptoms of an error, it is sometimes useful not to correct it right away, and to look for other blunders against the background of a known program behavior.
12) The most difficult to detect errors are induced ones, that is, those that were entered into the code when others were corrected.
8. Testing
Testing is the execution of a program for a set of test input values and a comparison of the results obtained with the expected ones.
The purpose of testing is to check and prove the correctness of the program. Otherwise, it reveals that there are errors in it. Testing itself does not show the location of the error and does not indicate its cause.
Principles of testing.
1) Test - a manually calculated example of program execution from the source data to the expected calculation results. These results are considered reference.
Full routing will be such testing, in which each linear section of the program will be passed at least when performing one test.
2) When running the program on test initial data, the results obtained should be checked against the reference data and analyze the difference, if any.
3) When developing tests, it is necessary to take into account not only correct, but also incorrect initial data.
4) We must check the program for undesirable side effects when setting some source data (division by zero, an attempt to read from a non-existent file, etc.).
5) Testing needs to be planned: pre-select what we control and how to do it better. Usually tests are planned at the stage of algorithmization or the choice of a numerical solution method. Moreover, when composing tests, we assume that there are errors in the program.
6) The more errors in the code we have already found, the greater the likelihood that we will find those not yet found.
A good call is a test that most likely should detect errors, and a good one is the one that found them.
9. Test Design
Tests are calculated manually, which means that they should be simple enough for this.
Tests should check each branch of the algorithm. If possible, of course. So the number and complexity of tests depends on the complexity of the program.
Tests are compiled before coding and debugging: during the development of an algorithm or even the compilation of a mathematical model.
Usually, to save time, they skip simpler tests first, and then more complicated ones.
Let's look at the task: you need to check whether the entered number falls within the range specified by the user.
program Example;
(************************************************** *****
* Task: check whether the entered number falls within *
* user-defined range *
****************************** *************************
var
min, max, A, tmp: real;
begin
writeln ('The program checks to see whether the entered user');
writeln ('values within the specified range');
writeln;
writeln ('Enter the lower limit of the range'); readln (min);
writeln ('Enter the upper limit of the range'); readln (max);
if min> max then begin
writeln ('You have confused the ranges, and I will change them');
tmp: = min;
min: = max;
max: = tmp;
end;
Repeat
writeln ('Enter the number to check (0 - end of work)'); readln (A);
if (A> = min) and (A <= max) then
writeln ('Number', A, 'falls in the range [', min, '..', max, ']')
else
writeln ('Number', A, 'misses the range [', min, '..', max, ']');
until A = 0;
writeln;
end.
If we proceed from the program's algorithm, we should compose the following tests:
entering the range boundaries
- min
- min> max
entering a number
- A 0)
- A> max (A <> 0)
- min <= A <= max (A <> 0)
- A = 0
As you can see, the program is very small, and quite a lot of tests are required to check all the branches of its algorithm.
10. Testing strategies
1) Testing the program as a "black box".
We only know about what the program does, but do not even think about its internal structure. We set the input data set, we get the results, we check with the reference ones.
In this case, we can detect all errors only if we have compiled tests for all possible data sets. Naturally, this is contrary to economic principles, and just silly enough.
A black box is convenient for testing small subroutines.
2) Testing the program as a "white box".
Here, before making a test, we study the logic of the program, its internal structure. Testing will be considered successful if it checks the program in all directions. However, as we have said, this requires a huge number of tests.
In practice, we, as always, share both principles.
3) Testing of modular structure programs.
We again return to the issue of structured programming. If you remember, programs are built from modules, not least to make them easy to debug and test. Indeed, we will test the structured program in parts. At the same time, we need to:
build a test suite;
combine modules for testing.
Such a combination can be built in two ways:
Step-by-step testing - we test each module, attaching it to the already tested ones. At the same time, we can connect parts of the program from top to bottom (top-down mode) or bottom-up (bottom-up).
Monolithic testing - each module is tested separately, and then a ready-made working program is formed from them and is already tested in its entirety.
To test a single module, you need a driver module (always one) and a plug-in module (there may be several of these).
The driver module contains fixed source data. It calls the module under test and displays (and possibly analyzes) the results.
A stub module is needed if there are calls from others in the module under test. Instead of this call, control is transferred to the stub module, and it already imitates the necessary actions.
Unfortunately, we are again confronted with the fact that drivers and stubs themselves can be a source of errors. Therefore, they should be created with great care.
Comments
To leave a comment
Quality Assurance
Terms: Quality Assurance