Bugs, the bug report and bug tracking systems

• Definition of a bug
• The difference between a disease (bug) and its manifestations (symptoms)
• The components of a bug, what's “inside”
• How to search for bugs correctly? - techniques and approaches
• What is bug Investigation?
• How to record bugs correctly
• The lifecycle of a bug
• types and kinds of bug-tracking systems

In programming a bug (English bug — original meanings: a bedbug, any insect, a virus) — is a slang word, usually denoting an error in a program or system, because of which the program produces unexpected behavior and, as a result, an unexpected outcome. Most bugs arise from mistakes made by the program's developers in its source code, or in its design. Some bugs also arise from incorrect operation of the compiler, which produces incorrect code. A program that contains a large number of bugs and/or bugs that seriously limit its operability is called unstable or, in slang, «glitchy», «buggy», «bugged», «flaky», «bug(g)ed»).

The term «bug» is usually used in relation to errors that manifest themselves at the program's runtime stage, as opposed to, for example, design errors or syntax errors. A report containing information about a bug is also called an error report or a problem report (English bug report). A report about a critical problem (English crash) that causes the program to terminate abnormally is called a crash report (English crash report).

«Bugs» are localized and eliminated during the process of testing and debugging a program.

Etymology of the term «bug»

An entry in a technical log

In the sense of an elusive technical fault, the word bug (English bug) was used long before the appearance of computers by staff of telegraph and telephone companies in relation to malfunctions of electrical equipment and radio engineering. In 1878 Thomas Edison wrote:

«It has been just so in all of my inventions. The first step is an intuition, and comes with a burst, then difficulties arise — this thing gives out and then that — «bugs» — as such little faults and difficulties are called — show themselves and months of intense watching, study and labor are requisite before commercial success or failure is certainly reached».

During the Second World War the word «bugs» was used for problems with radar electronics.

According to one version, in relation to a software error this term was first applied in 1946 by Grace Hopper, who was programming the Harvard Mark II computer at Harvard University (English) Russian.. Having traced an error in the program's operation down to an electromechanical relay of the machine, she found a moth stuck between the contacts. The extracted insect was taped into the technical logbook with an accompanying note: «First actual case of bug being found» (English First actual case of bug being found) .

The meaning and classification of software errors

Depending on the stage of software development at which an error is detected, the following are distinguished:

• syntax errors (recognized as such by the translator and making compilation impossible) — for example, the absence or mismatch of an opening and closing bracket;
• compiler warnings — for example, the use of an uninitialized variable. In this case the compiler may notice that the programmer is doing something unusual (probably incorrect), and reports it, but the programmer decides for themselves whether to ignore the message or not;
• runtime errors, semantic errors — for example, subtracting variables instead of adding them, or a segmentation fault.

By size:

• Showstoppers;
• Serious;
• Minor bugs;

By time of appearance:

• Constantly, on every run;
• Sometimes (a «floating» type);
• Only on the client's machine (depends on the client's local settings);

By place and direction:

• User interface errors;
• Error-handling system errors;
• Errors related to boundary conditions;
• Calculation errors;
• Flow-control errors;
• Data processing or interpretation errors;
• Race-condition errors;
• Load increase;
• Version control and identifier errors;
• Testing errors;

Depending on the nature of the error, the program and the runtime environment, an error may manifest itself immediately, or, on the contrary, remain unnoticed for a long time (for example, the Year 2038 problem).

Also an error may manifest itself as a vulnerability, making unauthorized access to the system or a DoS attack possible.

Varieties

• Bohrbug — an easily detectable, stable bug
• Heisenbug — a hard-to-detect bug that periodically disappears and changes its properties when an attempt is made to detect it
• Mandelbug — a bug with very complex, chaotic behavior
• Schrödinbug — a critical error that does not manifest itself until someone stumbles upon it in the source code, after which the program completely stops working

Finding and fixing errors

To debug a program (English debugging), software developers use special debugger programs (English debugger). For example, in the Windows operating system you can use the WinDbg program from the Microsoft Debugging Tools for Windows package. For GNU/Linux and a number of other UNIX-like operating systems there is the GDB (GNU Debugger) debugger.

Fig. an example of debugging an application (debugging)

Error reports

The bulk of bugs are usually debugged at the stage of compiling and testing the program. However, some errors still make it into the release and manifest themselves on end users' computers during the operation of the software. To improve the quality of software, special programs are used whose purpose is to catch an error in the target application, collect the necessary information about its symptoms and send a report over the internet to the developers of that software.

For example, the Windows operating system has the Dr. Watson utility built in, which by default catches errors in the user's applications and sends a report to a special Microsoft server. Similar libraries such as Breakpad and CrashRpt can also be cited as examples.

The consequences of the most notorious computer bugs in history.

• Errors in the software of the Therac-25 medical accelerator led to an overdose of radiation for several people.

Error tracking system

An error tracking system (English bug tracking system) — an application program designed to help software developers (programmers, testers, and others) keep track of and control errors and malfunctions found in programs, user wishes, and also to monitor the process of fixing these errors and the fulfillment or non-fulfillment of the wishes.

Composition of defect information

The main component of such a system is a database containing information about detected defects. This information may include:

• the defect number (identifier);
• a short description of the defect;
• who reported the defect;
• the date and time when the defect was discovered;
• the version of the product in which the defect was found;
• the severity (criticality) of the defect and the priority of the fix;
• a description of the steps to reveal the defect (reproducing the incorrect behavior of the program);
• the expected result and the actual result;
• who is responsible for fixing the defect;
• discussion of possible solutions and their consequences;
• the current state (status) of the defect;
• the version of the product in which the defect was fixed.

In addition, advanced systems provide the ability to attach files that help describe the problem (for example, a memory dump or a screenshot).

Bug Report

A bug or defect report - this is a document describing a situation or a sequence of actions that led to incorrect behavior of the object under test, indicating the causes and the expected result.

To obtain more detailed information about bug reports, we recommend the following information to your attention; having reviewed it you will get a comprehensive picture of the structure, the features of writing, and some other nuances needed to write good bug reports:

• The structure of a bug report
• Severity and Priority of a defect
• Writing bug reports
• The lifecycle of a bug

The main fields of a bug / defect report

Different defect management systems offer us different fields to fill in and different structures for describing defects. The table below is an attempt to show what, based on the experience we have gained, we recommend that you use as a bug report template.

Severity and priority of application errors

Different bug-tracking systems offer us different ways of describing the severity and priority of a bug report, but the meaning put into these fields stays unchanged. Everyone knows a bug tracker like Atlassian JIRA. In it, starting from some version, instead of using the Severity and Priority fields at the same time, only Priority was kept, which combined the properties of both fields: Originally, JIRA did have both a Priority and a Severity field. The Severity field was removed for a number of reasons...So those who are used to working with JIRA do not always understand the difference between these concepts, since they have had no experience using them together. Based on personal experience, I insist on separating these concepts, or more precisely on using both the Severity and Priority fields, since the meaning put into them is different:

Severity (Severity) - this is an attribute characterizing the impact of a defect on the operability of the application.

Priority (Priority) - this is an attribute indicating the order in which a task should be performed or a defect should be fixed. It can be said that this is a tool for a manager planning work. The higher the priority, the faster the defect needs to be fixed.

Grading of Defect Severity (Severity)

S1 Blocker (Blocker)
A blocking error that puts the application into a non-working state, as a result of which further work with the system under test or its key functions becomes impossible. Solving the problem is necessary for the further functioning of the system.

S2 Critical (Critical)
A critical error, incorrectly working key business logic, a hole in the security system, a problem that led to a temporary server crash or that puts some part of the system into a non-working state, with no possibility of solving the problem using other entry points. Solving the problem is necessary for further work with the key functions of the system under test.

S3 Major (Major)
A major error, part of the main business logic works incorrectly. The error is not critical, or there is a way to work with the function under test using other entry points.

S4 Minor (Minor)
A minor error that does not violate the business logic of the part of the application under test, an obvious user interface problem.

S5 Trivial (Trivial)
A trivial error that does not concern the application's business logic, a poorly reproducible problem, barely noticeable through the user interface, a problem with third-party libraries or services, a problem that has no effect at all on the overall quality of the product.

Grading of Defect Priority (Priority)

P1 High (High)
The error must be fixed as quickly as possible, since its presence is critical for the project.

P2 Medium (Medium)
The error must be fixed; its presence is not critical, but it requires a mandatory fix.

P3 Low (Low)
The error must be fixed; its presence is not critical, and it does not require an urgent fix.

The order of fixing errors by their priority:

High -> Medium -> Low

Requirements for the number of open bugs

We want to offer you the following approach to defining requirements for the number of open bugs:

• The presence of open defects P1, P2 and S1, S2 is considered unacceptable for the project. All such situations require an urgent fix and are put under the control of the project managers.
• The presence of a strictly limited number of open errors P3 and S3, S4, S5 is not critical for the project and is permitted in the released application. The number of open errors depends on the size of the project and the established quality criteria.

All requirements for open errors are agreed upon and documented at the stage of deciding on the quality of the product being developed. As an example of documenting such requirements, there is the item Test completion criteria in the test plan

Writing a bug report

A bug report - is a technical document, and in this regard we want to note that the language used to describe the problem should be technical. Correct terminology should be used when naming user interface elements (editbox, listbox, combobox, link, text area, button, menu, popup menu, title bar, system tray, etc.), user actions (click link, press the button, select menu item, etc.) and the results obtained (window is opened, error message is displayed, system crashed, etc.).

Requirements for the mandatory fields of a bug report

Note that the mandatory fields of a bug report are: short description (Bug Summary),severity (Severity), steps to reproduce (Steps to reproduce), result (Actual Result),expected result (Expected Result). Below are the requirements and examples for filling in these fields.

Short description

The name speaks for itself. In one sentence you need to fit the meaning of the whole bug report, namely: briefly and clearly, using correct terminology, say what does not work and where. For example:

1. The application hangs when trying to save a text file larger than 50Mb.
2. Data on the "Profile" form is not saved after clicking the "Save" button.

In addition, we suggest that you study the "Where? What? When?" principle, described on the pages of the"QA Nest" blog:

"What does this principle consist of?
Compose a sentence in which the facts of the defect are laid out in the following order:

• Where?: In what place of the user interface or of the software product's architecture the problem is located. Moreover, start the sentence with a noun, not a preposition.
• What?: What happens or does not happen according to the specification or your understanding of the normal operation of the software product. Here, indicate the presence or absence of the object of the problem, not its content (that is given in the description). If the content of the problem varies, all known variants are given in the description.
• When?: At what moment in the operation of the software product, upon the occurrence of what event, or under what conditions, the problem manifests itself.

Why must the order be exactly this?
In this form, unfamiliar defects are more convenient to sort by summary, as practice shows (since it is most likely among the defects of other engineers that a search for duplicates will be carried out). If you think differently - come up with your own order, but it must become uniform for all project members without exception, otherwise you will not achieve the necessary result."

Severity

In a nutshell, it can be noted that if a problem is found in the application's key functionality and after it occurs the application becomes completely unavailable, and further work with it is impossible, then it is blocking. Usually all blocking problems are found during the initial verification of a new version of the product (Build Verification Test, Smoke Test), since their presence does not allow testing to be carried out fully. If testing can be continued, then the severity of this defect will be critical. As for major, minor and trivial errors, the matter is fairly transparent and, in our opinion, does not require additional explanation.

Steps to Reproduce / Result / Expected Result

It is very important to clearly describe all the steps, mentioning all the input data (username, data for filling in the form) and intermediate results.

For example:

Steps to Reproduce
1. Log into the system: User Tester1, password xxxXXX
--> Login to the system performed
2. Click the Profile link
--> The Profile page opened
3. Enter New username: Tester2
4. Click the Save button
Result
An error appeared on the screen. The new username was not saved
Expected Result
The profile page reloaded. The new username value is saved.

Main mistakes when writing bug reports

Insufficient data provided
The same problem does not always manifest itself with all input values and under any user logged into the system, so it is strongly recommended to include all necessary data in the bug report

Determining severity
Very often there is either an overestimation or an underestimation of a defect's severity, which can lead to an incorrect order when solving the problem.

Description language
When describing a problem, incorrect terminology or complex turns of phrase are often used, which can mislead the person responsible for solving the problem.

Absence of an expected result
In cases where you have not indicated what the required behavior of the system should be, you waste the developer's time searching for this information, thereby slowing down the fix of the defect. You must indicate a clause in the requirements, a written test case, or your personal opinion, if this situation was not documented.

Filling in the fields of a bug report

The table described below shows the main fields of a bug report and the role of the employee responsible for filling in that field. Fields required to be filled in are highlighted in red:

Lifecycle (workflow) of a bug (defect)

As a rule, an error tracking system uses one variant or another of the «lifecycle» of an error, whose stage is determined by the current state, or status, in which the error is located.

A typical defect lifecycle:

1. New — the defect has been registered by the tester
2. Assigned — a person responsible for fixing the defect has been assigned
3. Resolved — the defect passes back into the tester's area of responsibility. As a rule, it is accompanied by a resolution, for example:
   • Fixed (the fix is included in version such-and-such)
   • Duplicate (repeats a defect that is already being worked on)
   • Not fixed (works according to the specification, has too low a priority, the fix is postponed to the next version, etc.)
   • Not reproducible (a request for additional information about the conditions under which the defect manifests itself).
4. Next the tester checks the fix, depending on which the defect either goes back into the Assigned status (if it is described as fixed but is not), or into the Closed status.
5. Reopened — the defect has been found again in another version.

The system may give the administrator the ability to configure which users can view and edit errors depending on their state, move them to another state, or delete them.

In a corporate environment, an error tracking system can be used to obtain reports showing the productivity of programmers in fixing errors. However, this approach often does not give sufficiently accurate results, because different errors have varying degrees of severity and complexity. At the same time, the severity of a problem has no direct relationship to the complexity of fixing the error.

Fig. The lifecycle of a bug

Before starting the description of the elementary lifecycle of a bug, we suggest looking at the following flowchart, showing the main statuses and possible transitions from status to status over the course of its existence:

Now let's move on to describing this diagram.

Suppose you found a bug and registered it in the bug-tracking system. According to our flowchart it will get the status “New”. The tester responsible for validating new bug reports, or the project coordinator (depending on the distribution of roles in your team), can move it into one of the following statuses:

• “Rejected”, if this bug is invalid or a duplicate, or it simply could not be reproduced
• “Deferred”, if this bug does not need to be fixed in this iteration
• “Open”, if fixing the bug is necessary

Let's now go through each of the options in order.

1. Rejected. In this case you can either argue about the fate of your bug report, changing the status to “Reopened” or close it - status “Closed”
2. Deferred. A bug report in the “Deferred” status can be moved to the “Open” status when a fix is needed, or to the “Closed” status, if it is no longer needed.
3. Open. It is in this state that the developer receives the bug report to fix. They can reject it (see item 1 for further actions) or fix the bug. A bug report in the “Fixed” status is passed to the tester for verification. If the problem is still reproducible, the “Reopened” status is set and the bug report is sent back to the developer for further work. If the fix was successful, the bug report is moved to the “Closed” status.

* * *

We want to note that this scheme is greatly simplified. For greater clarity, and possibly for the convenience of working on the project, you can add additional statuses and transitions, especially since modern bug-tracking systems allow this to be done. However, keep in mind that overly convoluted transition schemes and unnecessary statuses can significantly complicate life.

Note 1: in some bug-tracking systems a newly created bug report immediately gets the “Open” status without additional validation

Note 2: many bug-tracking systems allow reopening closed bugs, however I personally am against this practice, so I did not describe such a transition in the lifecycle presented above

Note 3: The lifecycle discussed above is based on the assumption that there is someone on the team responsible for assigning bug reports. If there is no such role on the project, then bugs are assigned by the developers themselves, and then, to avoid confusion, it makes sense to introduce one more intermediate status “In progress” (In progress), showing that this bug report has already been assigned and is at the fixing stage. An example implementation of such a lifecycle based on JIRA can be seen in the following figure:

Examples of error tracking systems

Freely distributed

• Redmine — is not itself an error tracking system, but many companies use it as one.
• BUGS - the Bug Genie http://www.thebuggenie.com/
• Bugzilla http://www.bugzilla.org/features/
• eTraxis https://www.etraxis.com/
• GNATS
• Launchpad
• Mantis bug tracking system
• Trac
• EmForge
• Picket
• Flyspray (site)
• DEVPROM

Proprietary

• Atlassian JIRA
• Bontq
• PVCS Tracker
• Project Kaiser
• TrackStudio Enterprise
• YouTrack

Other

• BugTracker.NET
• BugNet
• ClearQuest
• Intland CodeBeamer
• LifeTask.ru
• FlySpray
• StarTeam

See also

• [[b5190]]
• [[b5189]]
• [[b6116]]
• [[b766]]
• [[b9450]]
• [[b8636]]
• [[b6184]]
• Feature
• GIGO
• Patch

See also

• Bugs, the bug report and bug tracking systems
• Basic QA (Test) artifacts
• Quality assurance metrics and testing metrics
• 9.5 Bug and programming errors
• Causes of bugs: where do software errors come from?
• Try-catch exception handling: what's the difference between exceptions and errors?
• Types of errors and basic debugging principles.