Information Technology Reference
In-Depth Information
3.2 Test Materials
The digital cinema specification [15] provides guidance for selection of test mate-
rials for the subjective assessments' stimuli. Digital cinema is based on 2K or 4K
imagery, which is a significantly higher quality in terms of larger pixel counts per
image when compared to standard and high definition content, respectively. In or-
der to comply with the DCI specifications, the stimuli used in the assessment were
images taken from the DCI Standard Evaluation Material (StEM) [16]. From
these, six 2K images were selected. Because we only take into account the lumi-
nance component of images in this study, the luminance component was extracted
from each image resulting in six gray scale 2K images.
The subjective assessment was performed by examining a range of JPEG 2000
compression errors introduced by varying bit rates. In the design of a formal sub-
jective test, it is recommended to maintain a low number of compression condi-
tions in order to allow human subjects an easier completion of their evaluation
task. Accordingly, 8 different conditions were applied to create 8 processed im-
ages from each source image. The selected conditions covered the whole range of
quality levels, and the subjects were able to note the variation in quality from each
quality level to the next. This was verified prior to the subjective quality assess-
ment with a pilot test that involved expert viewers in order to conclude the selec-
tion of the final 8 bit rates. As a result of the pilot test, the selected bit rates were
in the range of 0.01 to 0.6 bits/pixel. To create 48 processed gray scale images, 6
source images were compressed using the KAKADU software version 6.0, with
the following settings: codeblock size of 64x64 (default), 5 decomposition levels
(default), and switched-off visual frequency weighting.
3.3 Test Methods and Conditions
There are several stimuli viewing sequence methods described in Recommenda-
tion ITU-R BT.500-11 [11]. They can be classified into two categories: single
stimulus (subjects are presented with a sequence of test images and are asked to
judge the quality of each test image) and double stimulus (subjects are presented
with the reference image and the test image before they are asked to judge the
quality of the test image). The presentation method of single stimulus is sequen-
tial, whereas the presentation method of double stimulus can be sequential and
simultaneous (side by side). The decision on which test method to use in a subjec-
tive assessment is crucial, because it has a high impact on the difficulty of the test
subjects' task. The pilot test prior to the main subjective assessment was also con-
ducted to compare sequential presentation and simultaneous presentation. Differ-
entiating between levels of high quality images requires a test method that
possesses a higher discriminative characteristic. Our pilot test indicated that the
simultaneous (side by side) presentation had a higher discriminative characteristic
than the sequential presentation order. Therefore, the subjective quality assess-
ment used the Simultaneous Double Stimulus test method, in which the subjects
are presented with the reference image and the distorted test image displayed side
by side on the screen. Figure 5 illustrates the display format in this method.
