Information Technology Reference
In-Depth Information
Image #
10
9
8
7
6
5
4
3
2
1
Excellent
Good
Fair
Poor
Bad
Fig. 6 Ten point quality scale and presentation structure of the test
3.4 Subjects
A proper evaluation of visual quality requires human subjects with good visual
acuity and high concentration, e.g. young persons such as university students. 29
subjects (10 female, 19 male) participated in the evaluation tests performed in this
work. 27 of them were university students. Some of the subjects were familiar
with image processing. Their age ranged from 21 to 32 years old. All subjects
re-ported that they had normal or corrected to normal vision.
3.5 Subjective Data Analysis
Before processing the resulting data, post-experiment subject screening was con-
ducted to exclude outliers using a method described by VQEG [17]. In addition to
using this method, the scores of each subject on reference images were also exam-
ined. As a result, one subject was excluded because he/she showed randomness
due to scoring low for the quality of reference images. Then the consistency level
for each of the remaining 28 subjects was verified by comparing his/her scores for
each of the 48 processed images to the corresponding mean scores of those images
over all subjects. The consistency level was quantified using Pearson's correlation
coefficient r, and if the r value for one subject was below 0.75, this subject was
excluded [17]. Here, the value of r for each subject was ≥ 0.9. Hence, data from all
remaining 28 subjects was considered.
All data was then processed to obtain the Mean Opinion Score (MOS) by aver-
aging the votes for all subjects. Figure 7 illustrates the MOS results. In addition,
the Standard Deviation and the 95% Confidence Intervals (CI) were computed
(based on a normal distribution assumption).
The behavior of a codec is generally content dependent, and this can be ob-
served in Fig. 7. As an example, for the lowest bit rate subjects scored higher for
Images 1 and 5 when compared to other images; these two images show a close up
face, which typically has low spatial complexity characteristics. Furthermore, Im-
age 2, which depicts a crowd and has high spatial complexity, tends to have the
lowest score of all the images except for the highest bit rate.
Search WWH ::




Custom Search