6.5 Redundancy of information. Coefficient of stochasticity

Let us write down the sequential values ​​of the information entropy, taking into account the increasingly extensive correlation for the Russian language

Let's return to the texts presented in tab. 1. previous paragraph

Using the function it can be calculated that during the transition from phrase 1 to phrase 4, the entropy of the text decreased by about 5 times. In phrase 2, the uncertainty of the appearance of each letter is approximately 4 bits per letter. In phrase 4, uncertainty is 1 bit per letter. This is because the structure of the real text contains information about all the grammatical and phonetic rules of the Russian language. The difference between the entropy of real text = 1 bit / letter and the maximum entropy of the phrase I _max = 5 bits / letter is the amount of information contained in the rules.

Language is characterized by redundancy of information that can be determined by the formula

, (one)

For the Russian language, the redundancy for correlations n = 0, 1, 2, 3 is

However, it is possible to look at the redundancy of information in a different way, namely: the study of messages from texts of telegrams, codes, etc. showed that all of them have redundant information, which can also not be transmitted via communication channels. But what is redundant for communication channels is not at all superfluous for the language itself. It is the redundant information accumulated in the aggregate of all grammatical and phonetic rules that made the language a language. Function reflected in itself the whole process of ordering as a gradual transition from the equality of all probabilities to their significant difference, when the probability of the letter “O” increases to 0.09, and the letter “F” drops to 0.0002.

Redundancy of the text has a very wide application in practice. Packers (archivers), well-known among computer users, compress files just due to the redundancy of information.

If the process of organizing the text continues to continue, then it is possible to predict what will eventually work out. Senseless phrase 1 received a semblance of familiar texts, when different letters began to have different probabilities. But the more probabilities of some letters, the less likelihood of the others due to the condition : the advantage of one letter is paid by the lack of rights of others.

We continue this process further, and then one letter of some kind, for example A, will take away all the rights, i.e. W _A = 1, and the likelihood of the remaining letters W _i = 0. Thus, the ordering brought to the end leads to a strange text Type AAAA ..., i.e. degeneration occurs.

However, in some cases, such a text may be useful: it can serve as a signal of some event. For example, the message that a cargo has arrived, a person, etc. As long as the probabilities of the letters are the same, the text is meaninglessly chaotic; probabilities of the appearance of letters are different - a certain order appears in the text; We continue this process further and the text turns into a repetition of one letter. Avoiding chaos, the text rushes to simplify, not carrying information. But at first glance, this seems to be a much more complicated matter: there are such phenomena in the world that resemble one-letter text, but, nevertheless, such information is useful. For example, the body responds stereotypically to some of the effects of the external environment: they felt pain when they burned — they refracted their hands reflexively. The same movement is performed for the same team, regardless of whether the object is heated to 70, 100 or 500 degrees. As long as there are different probabilities, the answers are varied; when it all comes down to one letter, the system gives the same answer to any requests. This is a hard deterministic system. An example is the Solar System, where the motions of the planets are predetermined for an infinitely long time. But this system was once entropic and unpredictable, a chaos of particles moving in space.

A rigidly deterministic system arose from chaos, since the conditions for the existence of a system are also strictly defined. It can be expected that in the distant future, having exhausted the energy, the Sun will go out, or some cosmic collision will occur, and the Solar System will again turn into a chaos of particles.

To a large extent, a deterministic system can be considered a heart, whose task is to rhythmically contract, unclench, and drive blood. When the external conditions change, the rhythm of the heart can be broken, and here the nervous system and other organs (the hormonal system) are created to adjust the process. Without them, when conditions change, the heart will not be able to perform its functions and will become unsuitable as text from one letter A, when it is not even clear where to send this primitive command.

To identify patterns of interaction of information with entropy, a convenient system turned out to be a written text that reflects a more complex system — the human language. Note that the study of various languages ​​showed that they all have similar statistical characteristics.

Our language is a flexible, mobile, easily adaptable system in different conditions. In language, there is a certain proportion of unpredictability, the share of "entropy." Consider this question in more detail.

We denote by the redundant predictable information that allows you to read the text in the absence of letters.

Magnitude I _n is equal to the difference between the entropy of the real text. = 1 bit per letter and maximum entropy, for example, phrases 1 (Table 1) I ₀ = 5 bits per letter, i.e.

I = I ₀ - = 5 - 1 = 4 bits per letter.

This is the amount of information contained in the grammatical and phonetic rules that govern the real texts. This amount of information I = 4 bits / letter is enough to turn abracadabra (phrase 1) into meaningful text.

Let G denote the stochasticity coefficient equal to

. (2)

Comparing formulas (1) and (2), we find the connection between the redundancy of information R and the coefficient of stochasticity G

.

So, for plain text, G = 0.25; the text from one "A" has zero entropy, i.e. = 0, therefore G = 0. Text with maximum entropy I = I ₀ - = 0 does not obey the rules and G = . In official documents there is a more rigid determinism than in literary texts, and G <0.25.

With G = 0, nothing new can be reported, with G = it is impossible to understand anything, because the flow of words collapses, the optimal ratio of unpredictability (entropy) and determination (rules) is G = 0.25. This is the result of a long evolution of the language.

Studies on the entropy of the combination of sounds in musical works have revealed the optimal ratio of determinism and stochasticity, and it is also G ~ 0.25.

The pictures also follow the rule of the golden mean. If the picture carries a strict classical laws, then their G is close to zero. On the contrary, in creations without thinking about G increases, in such works it is not always possible to catch the meaning.

In conclusion, we note that for real cases the G factor is very difficult to determine, here you can only talk about trends (G = 0 or G = 5). ), the final court can make on an intuitive level experts.

The reasoning about the stochasticity coefficient is applicable to many social phenomena - the architecture of cities, the fashion phenomenon, the human psyche, etc. Mental comfort, a feeling of happiness, is apparently determined by how well the individual’s living conditions correspond to the needs of his psyche. The latter is both inherent in it, and is shaped by its upbringing and can also be characterized by criterion G. Human nature is contraindicated as excessive bravado spontaneity (originality), and excessive determinism, subject to the requirements of benefit and convenience. High mental spontaneity leads to moral instability, anxiety, deprives the ability to work concentrated. Overly deterministic psyche kills creativity, deprives artistry. The task of psychiatry is to create an environment capable of determining too spontaneous psyche or, on the contrary, to “stir up” spontaneity with the help of emotions. Nature gives man the freedom to choose, allowing the suddenly found successful solution to find an unexpected turn.

Probably, the optimal stochasticity ratio G _opt at all levels of the organization — in an inert, living, and social nature — should be formed in any situation.

The purpose of man on earth is only in one continuity of the process of achievement, in other words, in life itself, and not in the goal itself, which is the formula "two and two is four." But “two and two and four” is no longer life, but the beginning of death.