7.1. SUBJECTIVE IMAGE QUALITY ASSESSMENT

Currently, the most common and most reliable way to determine the quality of images is a subjective examination [1-7]. In some cases, non-expert observers are invited as experts. Their estimates determine the quality of the image exactly as perceived by the “average” observer. In addition, experiments are carried out with specialists with experience in image processing, from which we should expect more reasonable estimates of quality. It is assumed that experienced observers notice small image errors that a non-specialist can overlook.

There are two types of expert assessments: absolute and comparative. In the first case, the observer must assess the quality of the image on a predetermined scale. In some methodologies, the assessment process is facilitated by the fact that the observer is also provided with a set of reference images. There are other methods when an observer is forced to make a decision only on the basis of his own experience. In comparative assessments, the observer must rank a set of specific images, that is, arrange them in a row in descending order of quality.

A common type of rating scale is the “overall quality” scale, when images are assigned scores that correspond to grades from “unsatisfactory” to “excellent” given in Table. 7.1.1 [1]. In practice, each image comes to the observer, and he puts the score of the category that, in his opinion, best corresponds to the quality of the image. A variant of this scale is the scale of the pole in a group, ”or the relative scale (Table 7.1.2), when the observer assesses the quality of the image, comparing it with all other images belonging to the group in question [2]. Another common rating scale is the error scale, according to which the observer must evaluate in points the degree of distortion, ranging from "imperceptible" to "highly undesirable." This scale, compiled by Mertz, Fowler and Christopher [3], is given in Table. 7.1.3. Several other quality assessment scales are described in the work of Pearson [1].

The results of expert assessments are usually expressed using the average score, defined as

, (7.1.1)

Where - the number of images related to category, a - corresponding to her score. It is believed that in order to obtain a reliable assessment of image quality, it is necessary to interview at least twenty observers. One of the difficulties associated with scoring is the possible non-linearity of the scale. In fig. 7.1.1 for comparison, the absolute (general) quality scale (Table 7.1.1), the error scale (Table 7.1.3) and one more error scale, consisting of three gradations, are placed. The comparison of the scales is made on the basis of subjective assessments [1].

Table 7.1.1. Overall quality scale (absolute scale)

Table 7.1.2. Group space scale (relative scale)

Table 7.1.8. Scale of errors

It should be emphasized that the results of subjective expert assessments are influenced by the nature of the images under consideration and the environment (experimental conditions). If the observer has seen similar images, he is inclined to more strictly assess the errors, since he has a well-established idea about the structure of the image. On the other hand, in an unfamiliar image, the distortions may go unnoticed until the observer has them indicated. It is clear that the experimental conditions should as far as possible correspond to the conditions of observation in a real situation. In addition, expert judgment should be used with caution if the conditions of observation have changed. Thus, the image quality on the screen of a conventional television monitor can be regarded as “good”, with “barely noticeable” distortions. However, if the same image is presented in the form of a photograph taken with the help of high-quality equipment, then all the errors previously hidden by the non-linearities of the television device may suddenly become very noticeable.

Fig. 7.1.1. Comparison of quality scales and errors used for the subjective assessment of images.