Lecture
To better understand approaches to software testing, you need, of course, to know what kinds and types of testing exist in principle. Let's start by looking at the basic types of testing that define a high-level classification of tests.
The highest level in the hierarchy of testing approaches is the concept of a type, which can cover several related testing techniques at once. In other words, a single type of testing can correspond to several of its kinds. Let's begin by considering a few types of testing that differ in knowledge of the internal structure of the object under test.
Black box — a term used to denote a system whose internal structure and mechanism of operation are very complex, unknown, or unimportant for the purposes of a given task. «The black box method» — a method of studying such systems in which, instead of examining the properties and relationships of a system's component parts, one studies the reaction of the system as a whole to changing conditions. The black box approach took shape in the exact sciences (in cybernetics, systems engineering and physics) in the 1920—1940s and was later adopted by other sciences (for example, by behavioral psychology).
Black-box testing or behavioral testing — a strategy (method) of testing the functional behavior of an object (program, system) from the point of view of the outside world, in which no knowledge of the internal structure (code) of the object under test is used. In other words, black-box testing is carried out by testers who do not have access to the application's source code. A strategy refers to the systematic methods of selecting and creating tests for a test suite. The strategy of behavioral testing proceeds from technical requirements and their specifications.
Gray-box testing is a combination of white-box testing and black-box testing. The goal of this testing is to find defects related to incorrect structure or incorrect use of applications.
In mathematics, statistics and computer modeling, a gray box model combines a partial theoretical structure with data to complete the model. The theoretical structure can range from information about the smoothness of the results to models that only need parameter values from data or existing literature. Thus, almost all models are «gray box» models, as opposed to « black box» models, where no model form is assumed, or « white box» models, which are purely theoretical. Some models take a particular form, such as linear regression or a neural network. They have special methods of analysis. In particular, linear regression methods are far more efficient than most nonlinear methods. A model can be deterministic or stochastic (i.e., contain random components) depending on its intended use.
White-box testing (eng. white-box testing), also glass-box testing (eng. glass-box testing), structural testing (eng. structural testing) — testing that takes into account the internal mechanisms of a system or component (ISO/IEC/IEEE 24765).
It usually includes testing of branches, paths, statements (see code coverage). During testing, inputs are chosen to execute different parts of the code and expected results are determined. This resembles in-circuit testing (eng.).
Black Box black-box testing method
Summary: We don't know how the system under test is built.
Black-box testing, also known as specification-based testing or behavior testing, is a testing technique based on working exclusively with the external interfaces of the system under test.
According to ISTQB:
Why exactly «black box»? For the tester, the program under test is like a black opaque box whose contents they cannot see. The goal of this technique is to find errors in the following categories:
Thus, we have no idea about the structure and internal design of the system. We need to concentrate on what the program does, not on how it does it.
Example of black-box testing:
A tester tests a website without knowing the specifics of its implementation, using only the input fields and buttons provided by the developer. The source of the expected result is the specification.
Since this is a type of testing, by definition it can include other kinds of it. Black-box testing can be either functional or non-functional. Functional testing involves checking the operation of the system's functions, while non-functional testing checks, accordingly, the general characteristics of our program.
The black-box technique is applicable at all levels of testing (from unit to acceptance), for which a specification exists. For example, when carrying out system or integration testing, requirements or a functional specification will form the basis for writing test cases.
Test design techniques based on the use of the black box include:
Advantages of the black-box method:
Disadvantages of the black-box method:
The opposite of the black-box technique is white-box testing, which will be discussed below.
Typical black-box test development methods include:
Summary: We know all the implementation details of the program under test.
White-box testing (also: transparent, open, glass-box; code-based or structural testing) – a software testing method that assumes the internal structure/design/implementation of the system is known to the tester. We choose input values based on knowledge of the code that will process them. In the same way, we know what the result of that processing should be. Knowledge of all the features of the program under test and its implementation is mandatory for this technique. White-box testing means going deeper into the internal structure of the system, beyond its external interfaces.
According to ISTQB:
white-box testing is:
Why «white box»? For the tester, the program under test is a transparent box whose contents they can see perfectly well.
Example of white-box testing:
A tester, who is usually a programmer, studies the implementation of the code of an input field on a web page, determines all the provided (both correct and incorrect) and non-provided user inputs, and compares the actual result of the program's execution with the expected one. In this case, the expected result is determined precisely by how the program's code is supposed to work.
White-box testing is similar to the work of a mechanic who studies a car's engine to understand why it won't start.
The white-box technique is applicable at different levels of testing – from unit to system level, but it is mainly used for the implementation of unit testing of a component by its author.
Advantages of the white-box method:
Disadvantages of the white-box method:
Comparison of Black Box and White Box
Summary: We know only some of the implementation details of the system under test.
Gray-box testing is a software testing method that involves a combination of White Box and Black Box approaches. That is, the internal structure of the program is known to us only partially. It assumes, for example, access to the internal structure and algorithms of the software's operation in order to write the most effective test cases, but the testing itself is carried out using the black-box technique, that is, from the user's point of view.
This testing technique is also called the translucent box method: we see some things, and we don't see others.
Example of using the gray-box method:
The tester studies the program's code in order to better understand the principles of its operation and to explore possible paths of its execution. This knowledge will help write a test case that is sure to check a specific piece of functionality.
The gray-box technique is applicable at different levels of testing – from unit to system level, but it is mainly applied at the integration level to check the interaction of different modules of the program.
Cem Kaner defines «gray-box testing as involving inputs and outputs, but the test design is based on information about the code or the operation of the program that is normally not visible to the tester». The methods of gray-box testing include:
Comments