Testing based on entity state diagrams

Having some experience in testing, I found that analysts and programmers often do not pay attention to incorrect transitions between states of entities. What does this mean in practice? For example, you can delete what has already been deleted, edit what has already been sent, etc. Such actions can lead to unhandled exceptions, including errors at the database level. Why do you need to handle such errors? At a minimum, it’s not good to show the user information about the logical structure of the database. Such errors often occur in multi-user systems, where several users can edit the same object. In this article, I will discuss how to design tests to test transitions between object states and how to quickly estimate the costs of such testing.

Entity State Requirements

In most cases, there are no explicit requirements for the states of the entity, and the tester has to formulate them himself, selecting the necessary information from the use cases / user stories. If the requirements are still there, they look something like this:


Well, if you have such diagrams. But if not, that's okay too, you can build them yourself according to the available information about possible operations on the object.

Example: Requirement State Mapping

Requirement 1: immediately after receiving messages from the mail server, they are marked as unread.
Requirement 2: when a user clicks on a message to open its content, the message becomes read.
Requirement 3: the user can mark previously read messages as unread.
Requirement 4: the user can delete messages (both read and unread); you cannot return messages after deletion.

According to these requirements, you can get the following message diagram:

How to design tests on state diagrams

Simple positive tests on state diagrams are a check of valid transitions between states, i.e. the ability to perform actions indicated by arrows. Negative tests - check the impossibility of performing any other actions. Thus, it is easy to calculate the number of necessary positive tests - the number of arrows in the diagram. The number of negative tests is also easy to calculate:

T ^- = N ² -T ⁺ ,

where T ^- is the number of negative tests, N is the number of states, T ⁺ is the number of positive tests.

The formula above is easily obtained from the matrix representation of the state diagram. Such a representation is constructed as follows:

• the matrix is ​​square, the number of rows and columns is equal to the number of states on the diagram;
• rows will indicate outgoing arrows, columns - incoming;
• if the arrow goes out of state 1 and enters state 2, + should be placed at the intersection of line 1 and column 2.

Thus, the number of pluses in the matrix corresponds to positive tests, and empty cells to negative ones. Their number is calculated according to the formula above.

The most difficult thing when testing states is not to count the number of tests or forget nothing (all tests are easily taken into account using the matrix), but to design real scenarios that correspond to negative tests.

According to this data representation, it is easy to give an estimate for testing, especially if time metrics are collected: the average time to write a test and the average time to complete it is enough to multiply each indicator by N ² and add them.

Example: tests for a message state diagram

Design the matrix representation for the following state diagram:



Thus, for this diagram there will be four simple positive tests and five negative ones.
If you design negative tests for your webmail client, they will look something like this:

- Not read - Not read:

1. View the list of messages on two browser tabs.
2. Find the read message and mark it unread on the first tab.
3. Without updating the second tab, mark the message selected in Step 2 with an unread message.

Expected result: the message on both tabs is marked as unread, in Step 3 there is no error message.

- Deleted - Deleted:

1. View the list of messages on two browser tabs.
2. Find an arbitrary message and delete it on the first tab.
3. Without updating the second tab, delete the message selected in Step 2.

Expected result: the message on both tabs is deleted (missing from the list of messages), in Step 3 there is no error message.

Designing such tests is the hardest part. It is possible that not all the empty cells in the table above will be able to come up with scenarios (although in this example it is possible). However, this is the easiest way to take into account all the tests for a state diagram.

Testing will obviously consist of two stages:

• check that positive tests can be performed correctly (existing state transitions);
• make sure there are no other transitions (negative tests).

Difficult tests on the state diagram

Simple positive tests are, as described above, the verification of one correct transition between two states. Thus, we can formulate the simplest criterion for covering a diagram with tests: all existing transitions between states have been tested, and the object has visited each state at least once. But you can design more complex scenario tests, consisting of more than one transition. Such tests can be further used for load testing. Similarly, you can set stronger test coverage criteria: all possible pairs of state transitions, triples, etc. are checked. (up to N-1, where N is the number of states of the object). Such a coating is called a Chow transition coating or an N-1 transition coating. Accordingly, the weakest coating will cover 0 transitions.

Other applications of state diagrams

The use of state diagrams, described above, is not entirely "classic." In the “classical” form, a state diagram represents possible states in which a program can be (for example, all possible screens) and transitions between them (respectively, ways to get from one screen to another). However, such a diagram may be too large for exhaustive testing (by any definition of coverage), so it may sometimes be advisable to combine several states into one and design higher-level tests that can be detailed if necessary.

Test design can be automated if the state diagram is specified using one of the formal specification languages ​​(VDM, Z, B, etc.). There are universal solutions for the formal approach to the drafting of specifications and, accordingly, testing on the basis of such models (uniTESK). But even if such tools are not used, the necessary test design can be done manually if the state diagram is not too large.