You get a bonus - 1 coin for daily activity. Now you have 1 coin

18.3. Entropy of a complex system. Entropy addition theorem

Lecture



In practice, it is often necessary to determine the entropy for a complex system obtained by combining two or more simple systems.

Under the union of the two systems   18.3.  Entropy of a complex system.  Entropy addition theorem and   18.3.  Entropy of a complex system.  Entropy addition theorem with possible states   18.3.  Entropy of a complex system.  Entropy addition theorem complex system is understood   18.3.  Entropy of a complex system.  Entropy addition theorem whose state   18.3.  Entropy of a complex system.  Entropy addition theorem represent all possible combinations of states   18.3.  Entropy of a complex system.  Entropy addition theorem systems   18.3.  Entropy of a complex system.  Entropy addition theorem and   18.3.  Entropy of a complex system.  Entropy addition theorem .

Obviously, the number of possible system states   18.3.  Entropy of a complex system.  Entropy addition theorem equally   18.3.  Entropy of a complex system.  Entropy addition theorem . Denote   18.3.  Entropy of a complex system.  Entropy addition theorem probability that the system   18.3.  Entropy of a complex system.  Entropy addition theorem will be able to   18.3.  Entropy of a complex system.  Entropy addition theorem :

  18.3.  Entropy of a complex system.  Entropy addition theorem . (18.3.1)

Probabilities   18.3.  Entropy of a complex system.  Entropy addition theorem conveniently arranged in a table (matrix)

  18.3.  Entropy of a complex system.  Entropy addition theorem

  18.3.  Entropy of a complex system.  Entropy addition theorem

  18.3.  Entropy of a complex system.  Entropy addition theorem

  18.3.  Entropy of a complex system.  Entropy addition theorem

  18.3.  Entropy of a complex system.  Entropy addition theorem

  18.3.  Entropy of a complex system.  Entropy addition theorem

  18.3.  Entropy of a complex system.  Entropy addition theorem

  18.3.  Entropy of a complex system.  Entropy addition theorem

  18.3.  Entropy of a complex system.  Entropy addition theorem

  18.3.  Entropy of a complex system.  Entropy addition theorem

  18.3.  Entropy of a complex system.  Entropy addition theorem

  18.3.  Entropy of a complex system.  Entropy addition theorem

  18.3.  Entropy of a complex system.  Entropy addition theorem

  18.3.  Entropy of a complex system.  Entropy addition theorem

  18.3.  Entropy of a complex system.  Entropy addition theorem

  18.3.  Entropy of a complex system.  Entropy addition theorem

  18.3.  Entropy of a complex system.  Entropy addition theorem

  18.3.  Entropy of a complex system.  Entropy addition theorem

  18.3.  Entropy of a complex system.  Entropy addition theorem

  18.3.  Entropy of a complex system.  Entropy addition theorem   18.3.  Entropy of a complex system.  Entropy addition theorem   18.3.  Entropy of a complex system.  Entropy addition theorem

  18.3.  Entropy of a complex system.  Entropy addition theorem

  18.3.  Entropy of a complex system.  Entropy addition theorem

  18.3.  Entropy of a complex system.  Entropy addition theorem

  18.3.  Entropy of a complex system.  Entropy addition theorem

  18.3.  Entropy of a complex system.  Entropy addition theorem

  18.3.  Entropy of a complex system.  Entropy addition theorem

Find the entropy of a complex system. By definition, it is equal to the sum of the products of the probabilities of all its possible states to their logarithms with the opposite sign:

  18.3.  Entropy of a complex system.  Entropy addition theorem (18.3.2)

or, in other designations:

  18.3.  Entropy of a complex system.  Entropy addition theorem . (18.3.2 ')

The entropy of a complex system, like the entropy simple, can also be written in the form of a mathematical expectation:

  18.3.  Entropy of a complex system.  Entropy addition theorem , (18.3.3)

Where   18.3.  Entropy of a complex system.  Entropy addition theorem - the logarithm of the probability of the state of the system, considered as a random variable (state function).

Suppose the systems   18.3.  Entropy of a complex system.  Entropy addition theorem and   18.3.  Entropy of a complex system.  Entropy addition theorem are independent, that is, they take their states independently of one another, and we calculate under this assumption the entropy of a complex system. By the probability multiplication theorem for independent events

  18.3.  Entropy of a complex system.  Entropy addition theorem ,

from where

  18.3.  Entropy of a complex system.  Entropy addition theorem .

Substituting in (18.3.3), we get

  18.3.  Entropy of a complex system.  Entropy addition theorem ,

or

  18.3.  Entropy of a complex system.  Entropy addition theorem , (18.3.4)

that is, when combining independent systems, their entropies are added.

The proven position is called the entropy addition theorem.

The entropy addition theorem can be easily generalized to an arbitrary number of independent systems:

  18.3.  Entropy of a complex system.  Entropy addition theorem . (18.3.5)

If the systems being combined are dependent, simple addition of entropy is no longer applicable. In this case, the entropy of a complex system is less than the sum of the entropies of its constituent parts. To find the entropy of a system composed of dependent elements, you need to introduce a new concept of conditional entropy.


Comments


To leave a comment
If you have any suggestion, idea, thanks or comment, feel free to write. We really value feedback and are glad to hear your opinion.
To reply

Algorithms

Terms: Algorithms