Technological Evaluation and Optimization of E-Learning Systems Components - E-Infrastructures and Technologies for Lifelong Learning - page 153

1. INTRODUCTION: THE PROBLEM
OF MULTIPLE CRITERIA
EVALUATION AND OPTIMIZATION
OF LEARNING SOFTWARE

Function”), including the learning software evalu-
ation criteria , their ratings (values) and weights .
Expert evaluation is referred to a multiple
criteria evaluation of the learning software that is
aimed at the selection of the best alternative based
on score-ranking results. According to Dzemyda
& Saltenis (1994), if a set of decision alternatives
is assumed to be predefined, fixed and finite, then
the decision problem can be formulated as a task
of finding the optimal alternative or ranking the
various alternatives. In practice, usually experts
(decision makers) have to deal with the problem
of making the optimal decision in the multiple
criteria situation where the objectives are often
conflicting. In this case, according to Dzemyda
and Saltenis (1994), “an optimal decision is the
one that maximizes the decision maker's utility”.
The authors of the Chapter apply here one the
software engineering principles which claims that
one should evaluate the software using two differ-
ent groups/types of evaluation criteria: 'internal
quality' and 'quality in use'. According to Gas-
perovic and Caplinskas (2006), 'internal quality'
is a descriptive characteristic that describes the
quality of software irrespective of any particular
context of use, and 'quality in use' is an evalua-
tive characteristic of software obtained by making
a judgment based on criteria that determine the
worthiness of software for a particular project
or user/group. According to Gasperovic and Ca-
plinskas (2006), it is impossible to evaluate the
quality in use without knowing the characteristics
of internal quality.
The rest of the chapter is organized as follows.
The next section presents the literature review
and a short analysis of the existing technological
evaluation models (i.e., sets of evaluation criteria)
and methods for evaluation of LOs, LORs and
VLEs. Then, multiple criteria evaluation and op-
timization of learning software for the particular
learner needs are described. The fourth section
offers further research trends whist conclusions
are provided in the fifth section.

The problem of evaluation and optimization of
the technological quality of e-learning systems
components, i.e. Learning Objects (LOs), Learn-
ing Object Repositories (LORs) and Virtual Learn-
ing Environments (VLEs), is high on the agenda
of the European research and education system.
Evaluation can be characterized as the process
by which people make judgements about value
and worth. However, in the context of learning
technologies, this judgement process is complex
and often controversial. Although the notion of
evaluation is rooted in a relatively simple concept,
the process of judging the value of learning tech-
nology is complex and challenging (Oliver, 2000).
Different scientific methods are used for evalu-
ating the quality of e-learning systems components
(i.e. learning software). The chapter is aimed to
consider the problems of expert evaluation of the
technological quality of LOs, LORs and VLEs.
The basic notions, principles and methods ap-
plied in the Chapter are as follows: LO is referred
to any digital resource that can be reused to support
learning (Wiley, 2000). LORs are considered here
as properly constituted systems (i.e. organized
LOs collections) consisting of LOs, their metadata
and tools / services to manage them (Kurilovas,
2007). Metadata is referred to structured data about
data (Duval et al., 2002). VLEs are considered
as specific information systems which provide
a possibility to create and use different learning
scenarios and methods.
Quality evaluation is defined as “the system-
atic examination of the extent to which an entity
(part, product, service or organization) is capable
of meeting specified requirements” (ISO/IEC
14598-1:1999).
Multiple criteria evaluation method is referred
here as the experts' additive utility function (e.g.
Equation 3 in section “Experts' Additive Utility