Evaluation - Advances in Personalized Web-Based Education - page 95

based on Likert scale with five responses ranging from “Very much” (5) to
“Not at all” (1), was used. The experiment included an experimental group (the
group of students which used the presented programming tutoring system) and
two control groups (students who had not used some programming tutoring sys-
tem). The average degrees that the students succeeded in two related with pro-
gramming lessons, which they taught after the use of the programming tutoring
system, were calculated and compared for each of the three groups.
• Learners' satisfaction about the system's adaptive responses to their
needs : For gathering this kind of information a questionnaire (Questionnaire E,
Appendix B) was used. The questions were close-ended based on Likert scale
with five responses ranging from “Very much” (5) to “Not at all” (1).
• The validity of the conclusions drawn by the student model concerning the
aspects of the students' characteristics : It is performed assessing the results of
the system's use in relation with the different backgrounds of the students. More
concretely, the students, who were used the presented programming tutoring
system, were divided into three categories according to their background knowl-
edge. These three categories are arts, science fields (other than computer
science) and computer science related fields. Learners, which have studies in
the field of human, social, political and educational sciences, belong to arts.
Learners, who have studies in the field of mathematic, physic, business and
economic sciences, belong to science fields. Learners, who have studies in
the field of computer, engineering and system sciences, belong to computer
science-related fields. It is obvious that students, who belong to the last cat-
egory, have a previous knowledge on computer programming. Thereby, their
progress should be better than the students of the other two categories. Also,
the particular learners should be advised to read a chapter fewer times than the
others. Furthermore, the learners, who belong to science fields, should have
a good learning pace, since their studies offer them a way of thinking that is
close to the programming logic. Whilst, the students of the arts' category should
be advised to read more times the learning material, until to learn the knowl-
edge domain, since they often have no idea about computer programming.
Consequently, for each background category, the average reading times of each
domain concept and the percentage of learners that are advised to return to a
previous concept in order to revise it were measured. Furthermore, those values
were compared in order to assess if the conclusions drawn by the system con-
cerning the characteristics of student model are valid.
• The validity of the adaptation decision making of the student model : The
evaluation of the particular criterion includes assessment of the system's adapta-
tion decisions about the return of a learner to a domain concept, which was con-
sidered as learner and/or assimilated, in order to revise it and about the inference
of the system that a learner should not read a particular domain concept at all.
The particular assessment is performed using a questionnaire and conducting an
experiment. In particular, a questionnaire (Questionnaire F, Appendix B) with
close-ended questions based on Likert scale with five responses ranging from
“Very much” (5) to “Not at all” (1), was used to ask the learners' opinion about