Graphics Reference
In-Depth Information
9.4.2
Subjective Quality Evaluation Test
This section reports the results of subjective quality evaluation conducted at EPFL's
MMSPG test laboratory, which fulfills the recommendations for the subjective eval-
uation of visual data issued by ITU-R BT.500 [
11
,
14
]. It is also worth noting that
the testing methodology performed in this section has benefited significantly from
the experience gained while conducting the subjective evaluation tests described
in [
8
].
9.4.2.1
Test Environment
The test room is equipped with a controlled lighting system with a 6,500 K color
temperature and an ambient luminance at 15 % of the maximum screen luminance,
whereas the color of all the background walls and curtains present in the test area
are in mid grey. The laboratory setup is intended to ensure the reproducibility of the
subjective tests results by avoiding unintended influence of external factors.
To display the test stimuli, two Eizo CG301W LCD monitors with a native
resolution of 2,560
1,600 pixels were used. The monitors were calibrated using
an X-Rite i1Display Pro color calibration device according to the following profile:
sRGB gamut, D65 white point, 120 cd/m
2
brightness, and minimum black level.
The experiment involved two subjects per monitor assessing the test material.
The subjects were seated in a row perpendicular to the center of the monitor, at a
distance of 2.2 times the picture height, roughly corresponding to a visual angle of
1 arc-minute between two adjacent pixels, as suggested in [
13
].
9.4.2.2
Test Methodology
The double stimulus impairment scale (DSIS Variant I) methodology as described
earlier was chosen for the testing. A five-grade impairment scale (5: Imperceptible,
4: Perceptible but not annoying, 3: Slightly annoying, 2: Annoying, 1: Very
annoying) was used. The subjects were presented with pairs of video sequences
(i.e., stimuli), where the first sequence was always a reference video (stimulus A)
and the second, the video to be evaluated (stimulus B). After the presentation of
each pair of sequences, a 5-s voting time followed. Subjects were asked to rate the
impairments of the second stimulus in relation to the first stimulus, and to express
these judgments in terms of the wordings used to define the rating scale.
9.4.2.3
Dataset
Five video sequences in Table
9.2
were used in the experiments, with different
visual characteristics, resolutions, and frame rates. All sequences were stored as
