You get a bonus - 1 coin for daily activity. Now you have 1 coin

Statistical Pattern Recognition Techniques

Lecture



Speaking of statistical methods of recognition, we assume the establishment of a relationship between the assignment of an object to a particular class (image) and the probability of error when solving this problem. In some cases, this comes down to determining the posterior probability of the object being in the image Statistical Pattern Recognition Techniques provided that the attributes of this object took on the values Statistical Pattern Recognition Techniques . Let's start with the Bayes decision rule. By Bayes formula

Statistical Pattern Recognition Techniques

Here Statistical Pattern Recognition Techniques - a priori probability of presentation to object recognition Statistical Pattern Recognition Techniques th image:

Statistical Pattern Recognition TechniquesStatistical Pattern Recognition Techniques .

For each Statistical Pattern Recognition Techniques

Statistical Pattern Recognition Techniques ,

with signs of continuous measurement scale

Statistical Pattern Recognition Techniques ,

with signs with a discrete measurement scale

Statistical Pattern Recognition Techniques .

At continuous values ​​of signs Statistical Pattern Recognition Techniques is a probability density function, with discrete - probability distribution.

Distributions describing different classes, as a rule, "intersect", that is, there are such characteristic values Statistical Pattern Recognition Techniques at which

Statistical Pattern Recognition Techniques .

In such cases, recognition errors are inevitable. Naturally, there are no interesting cases when these classes (images) in the chosen feature system Statistical Pattern Recognition Techniques indistinguishable (with equal a priori probabilities, solutions can be chosen by randomly assigning an object to one of the classes in an equiprobable manner).

In general, one should strive to choose the decision rules in such a way as to minimize the risk of loss in recognition.

The risk of loss is determined by two components: the probability of recognition errors and the magnitude of the "penalty" for these errors (losses). Matrix of recognition errors:

Statistical Pattern Recognition Techniques ,

Where Statistical Pattern Recognition Techniques - the probability of correct recognition;

Statistical Pattern Recognition Techniques - the probability of erroneous assignment of the object Statistical Pattern Recognition Techniques th image to Statistical Pattern Recognition Techniques -mu ( Statistical Pattern Recognition Techniques ).

Loss matrix

Statistical Pattern Recognition TechniquesStatistical Pattern Recognition Techniques ,

Where Statistical Pattern Recognition Techniques - "award" for the correct recognition;

Statistical Pattern Recognition Techniques - "penalty" for the erroneous assignment of an object Statistical Pattern Recognition Techniques th image to Statistical Pattern Recognition Techniques -mu ( Statistical Pattern Recognition Techniques ).

It is necessary to construct a decision rule in such a way as to ensure a minimum of expected losses (minimum of average risk). This rule is called Bayesian.

We divide the sign space Statistical Pattern Recognition Techniques on Statistical Pattern Recognition Techniques disjoint areas Statistical Pattern Recognition Techniques each of which corresponds to a particular image.

The average risk with hit implementations Statistical Pattern Recognition Techniques th image in the area of ​​other images is equal

Statistical Pattern Recognition Techniques , Statistical Pattern Recognition Techniques .

Here it is assumed that all components Statistical Pattern Recognition Techniques have a continuous scale of measurements (in this case it is not fundamental).

Magnitude Statistical Pattern Recognition Techniques can be called a conditional average risk (provided that an error is made in recognizing an object Statistical Pattern Recognition Techniques th image). The total (unconditional) average risk is determined by Statistical Pattern Recognition Techniques

Decision rules (partitioning methods Statistical Pattern Recognition Techniques on Statistical Pattern Recognition TechniquesStatistical Pattern Recognition Techniques ) form a multitude Statistical Pattern Recognition Techniques . The best (Bayesian) decisive rule is that which provides minimal average risk. Statistical Pattern Recognition Techniques where Statistical Pattern Recognition Techniques - the average risk in applying one of the decisive rules included in Statistical Pattern Recognition Techniques .

Consider the simplified case. Let be Statistical Pattern Recognition Techniques , but Statistical Pattern Recognition Techniques ( Statistical Pattern Recognition Techniques ). In this case, the Bayes decision rule ensures a minimum of the probability (average number) of recognition errors. Let be Statistical Pattern Recognition Techniques . The probability of an error of the first kind (the object of the 1st image is assigned to the second image)

Statistical Pattern Recognition Techniques ,

Where Statistical Pattern Recognition Techniques - probability of error of the second kind

Statistical Pattern Recognition Techniques .

Average errors

Statistical Pattern Recognition Techniques .

Because Statistical Pattern Recognition Techniques then Statistical Pattern Recognition Techniques and Statistical Pattern Recognition Techniques . Clearly at least Statistical Pattern Recognition Techniques will have a minimum if the integrand in the region Statistical Pattern Recognition Techniques will be strictly negative that is in Statistical Pattern Recognition TechniquesStatistical Pattern Recognition Techniques . In the area of Statistical Pattern Recognition Techniques the opposite inequality must be satisfied. This is the Bayes decision rule for the case under consideration. It can be written differently: Statistical Pattern Recognition Techniques ; magnitude Statistical Pattern Recognition Techniques considered as a function of Statistical Pattern Recognition Techniques called likelihood Statistical Pattern Recognition Techniques at this Statistical Pattern Recognition Techniques , but Statistical Pattern Recognition Techniques - likelihood ratio. Thus, the Bayes decision rule can be formulated as a recommendation to choose a solution. Statistical Pattern Recognition Techniques if the likelihood ratio exceeds a certain threshold value that does not depend on the observed Statistical Pattern Recognition Techniques .

Without special consideration, we note that if the number of recognized classes is more than two ( Statistical Pattern Recognition Techniques ), the decision in favor of the class (image) Statistical Pattern Recognition Techniques is accepted in the field Statistical Pattern Recognition Techniques in which for all Statistical Pattern Recognition TechniquesStatistical Pattern Recognition Techniques .

Sometimes with a low accuracy of a posteriori probability estimation (small volumes of a training sample), so-called randomized decision rules are used. They consist in the fact that an unknown object is attributed to a particular image not according to the maximum a posteriori probability, but in a random way, in accordance with the a posteriori probabilities of these images Statistical Pattern Recognition Techniques . This can be implemented, for example, in the manner shown in Fig. 18.

Statistical Pattern Recognition TechniquesStatistical Pattern Recognition TechniquesStatistical Pattern Recognition TechniquesStatistical Pattern Recognition TechniquesStatistical Pattern Recognition TechniquesStatistical Pattern Recognition Techniques

Statistical Pattern Recognition Techniques

Fig. 18. Illustration of randomized decision rule

After calculating the posterior probabilities of belonging of an unknown object with parameters Statistical Pattern Recognition Techniques each of the images Statistical Pattern Recognition Techniques , Statistical Pattern Recognition Techniques , a straight line length of the unit is divided into Statistical Pattern Recognition Techniques intervals with numerically equal lengths Statistical Pattern Recognition Techniques , and each interval is associated with this image. Then, using a random (pseudo-random) number sensor, uniformly distributed on Statistical Pattern Recognition Techniques , generate a number, determine the interval in which it fell, and assign the recognizable object to the image to which the given interval corresponds.

It is clear that such a decision rule cannot be better than Bayesian, but for large values ​​of the likelihood ratio is not much inferior to it, and in implementation it can be quite simple (for example, the nearest neighbor method, which will be discussed later).

Bayesian decision rule is implemented in computers mainly in two ways.

1. Direct calculation of a posteriori probabilities

Statistical Pattern Recognition Techniques ,

Where Statistical Pattern Recognition Techniques - vector of values ​​of parameters of a recognizable object and selection of a maximum The decision is made in favor of the image for which Statistical Pattern Recognition Techniques as much as possible. In other words, the Bayes decision rule is implemented by solving the problem Statistical Pattern Recognition Techniques .

If we go to further generalization and assume the existence of a loss matrix of a general form, then the conditional risk can be determined by the formula Statistical Pattern Recognition Techniques , Statistical Pattern Recognition Techniques . Here the first member defines the "encouragement" for correct recognition, and the second - the "punishment" for the mistake. The Bayesian decision rule in this case is to solve the problem.

Statistical Pattern Recognition Techniques

2. "Topographic" area definition Statistical Pattern Recognition Techniques which hit vector Statistical Pattern Recognition Techniques feature values ​​describing a recognizable object.

This approach is used in cases where the description of areas Statistical Pattern Recognition Techniques quite compact, and the procedure for determining the area in which it fell Statistical Pattern Recognition Techniques is simple. In other words, it is natural to use this approach when computationally it is more efficient (simpler) than direct calculation of a posteriori probabilities.

Statistical Pattern Recognition Techniques

Fig. 19. Bayesian decision rule
for normally distributed attributes
with equal covariance matrices

So, for example (we will not give the proof), if there are two classes, their prior probabilities are the same, Statistical Pattern Recognition Techniques and Statistical Pattern Recognition Techniques - normal distributions with the same covariance matrices (differ only in the vectors of the means), then the Bayesian dividing boundary is the hyperplane. It is remembered by the values ​​of the coefficients of the linear equation. When recognizing an object, the values ​​of attributes are substituted into the equation Statistical Pattern Recognition Techniques of this object and by the sign (plus or minus) of the obtained solution the object is referred to Statistical Pattern Recognition Techniques or Statistical Pattern Recognition Techniques (Fig. 19).

If y classes Statistical Pattern Recognition Techniques and Statistical Pattern Recognition Techniques covariance matrices Statistical Pattern Recognition Techniques and Statistical Pattern Recognition Techniques not only the same, but also diagonal, the Bayesian solution is to assign an object to that class, the Euclidean distance to the standard of which is minimal (Fig. 20).

Statistical Pattern Recognition Techniques

Fig. 20. Bayesian decision rule
for normally distributed attributes
with equal diagonal covariance matrices
(elements of the diagonals are the same)

Thus, we are convinced that some decision rules, previously considered by us as empirical (deterministic, heuristic), have a very clear statistical interpretation. Moreover, in some specific cases, they are statistically optimal. We will continue the list of similar examples by further consideration of statistical recognition methods.

We now turn to methods for estimating the distribution of values ​​of features of classes. Knowledge Statistical Pattern Recognition Techniques is the most universal information for solving problems of recognition by statistical methods. This information can be obtained in two ways:

define in advance Statistical Pattern Recognition Techniques for all Statistical Pattern Recognition Techniques and Statistical Pattern Recognition Techniques ;

define Statistical Pattern Recognition Techniques at each act of recognition of a particular object, the attributes of which have values Statistical Pattern Recognition Techniques .

Each of these approaches has its advantages and disadvantages, depending on the number of features, the size of the training sample, the availability of a priori information, etc.

Let's start with the local version (definitions Statistical Pattern Recognition Techniques in the vicinity of a recognizable object).


Comments


To leave a comment
If you have any suggestion, idea, thanks or comment, feel free to write. We really value feedback and are glad to hear your opinion.
To reply

Pattern recognition

Terms: Pattern recognition