Information Technology Reference
In-Depth Information
Table 3.6
(
continued
)
17
4
3
4
3
2
4
18
6
6
6
6
5
6
19
6
6
6
6
5
6
20
4
3
3
3
2
4
Table 3.7
Relative results of expert evaluations
Expert
Tutorials
1
2
3
4
5
6
1
0,1
0,1 0,05
0,1 0,05 0,1
2
0,05
0,1
0,05
0,15
0,2
0,05
3
0,15
0,2
0,3
0,2
0,15
0,15
4
0,2
0,2
0,1
0,15
0,15
0,2
5
0,15
0,25
0,2
0,2
0,35
0,15
6
0,35
0,15
0,3
0,2
0,1
0,35
Expert evaluations in the form of numbers of the verbal scale levels are given
in Table 3.6.
Relative results of expert evaluations within the limits of each level are
summarized in Table 3.7. Number of a line corresponds to number of the rating
scale level.
Six COSS's
i
are constructed, which formalize criteria of expert
evaluations of courseware correspondence to curricula. While constructing, data
of Table 3.7 and a method d
esc
ribed
in
§ 2.2 were used. Parameters of
membership functions
X
;
=
1
()
=
l
of term-sets COSS are summarized
in Table 3.8, and parameters of membership functions of the formalized expert
results — in Table 3.9.
Indexes of the general consistency of results are the following: the additive
index is equal to 0.218, the multiplicative index is equal to 0, that indicates
availability of significantly differing results. For comparison, the concordation
coefficient calculated within the expert ranking limits is equal to 0.07.
Elements of a matrix of a pairwise similarity of the formalized expert results
are presented below.
μ
x
;
;
i
=
1
1
il
Elements of a matrix of a pairwise similarity of the formalized expert results
1
0,57
0,75
0,54
0,48
1
0,57
1
0,61
0,84
0,79
0,57
0,75 0,61
1
0,59
0,59
0,75
0,54 0,84
0,59
1
0,75
0,54
0,48 0,79
0,59
0,75
1
0,48
1
0,57
0,75
0,54
0,48
1
