Experimental methods for determining the reliability of complex software.

There are direct experimental methods for determining the reliability of complex PS and forced methods.
Direct experimental methods for determining the reliability indicators of software systems are used under normal operating conditions. They require large values ​​of time to failure, so their cost is high. In addition, in such experiments it is difficult to guarantee the complete representativeness of the sample of the initial data, since Checks are determined by the specific conditions of use of this PS under test.
To diagnose and eliminate occasional rare failures, a service should be organized for registering all failures with full recording of situations in which the failure occurred. If the cause of the failure is not hardware, then you need to conduct in-depth testing of the functional component, which may contain a defect that caused the failure.
During the final acceptance and certification tests, for a fairly reliable determination of the reliability of the software, hours-long and many-day runs of the program complex functioning in a real or simulated external environment under conditions of wide variation of the initial data are organized. Such runs allow you to measure and record the achieved quality indicators and the degree of their compliance with the requirements of the technical specifications. If intensive testing of programs for a sufficiently long time does not lead to the detection of defects or errors, the PS is transferred to operation.
Forced tests of the reliability of software systems are carried out by increasing the intensity of distortions of the initial data and expanding the variation of their values, as well as by increasing the information flows and loading the PS above normal. For forced testing, the following test modes are used:
with complete distortion of the key parameters of each type of external information;
- with the limiting and critical values ​​of the parameters of each type of information;
- with limiting and critical combinations of the values ​​of various interacting parameters;
- at extremely large and small intensities of the total flow of external information;
- at extremely high intensities and the absence of each type of external information.
As a kind of forced testing, it is possible to consider testing and monitoring the results of the operation of the same PS with an increase in the number of test specimens and normal source data. This allows, depending on the number of copies of the PS, to extend test suites in the appropriate number of times and makes it possible to evaluate the time between failures for hundreds of thousands of hours.
The reliability of complex, critical PSs is significantly affected by memory overloads and performance during program execution. In order to detect failures due to these reasons, during the accelerated tests, testing is carried out at a high intensity of the input data.