Information Technology Reference
In-Depth Information
Evaluation
toward the course's effectiveness and the delivery
technology (Willis, 2008).
Summative evaluation
consists of tests designed
for criterion-related referenced items and providing
opportunities for feedback from the users. Revisions
are made as necessary (Colston, 2008). This is con-
ducted after instruction is completed and provides
a data base for course revision and future planning.
Following course completion, consider a summa-
tive evaluation session in which students informally
brainstorm ways to improve the course. Consider
having a local facilitator run the evaluation session
to encourage a more open discussion (Willis, 2008).
One purpose of evaluation is to determine if the
instructional methods and materials are accom-
plishing the established goals and objectives.
Implementation of instruction represents the
first real test of what has been developed. Try
to pre-test instruction on a small scale prior to
implementation. If this is not possible, the first
actual use will also serve as the “field test” for
determining effectiveness. This step consists of
two objectives (Willis, 2008):
1.
Develop an evaluation strategy
: Plan how
and when to evaluate the effectiveness of
the instruction.
2.
Collect and analyze evaluation data
:
Following implementation of the course/
materials, collect the evaluation data. Careful
analysis of these results will identify gaps or
weaknesses in the instructional process. It is
equally important to identify strengths and
successes. Results of the evaluation analysis
will provide a “springboard” from which to
develop the revision plan (Willis, 2008).
Formative evaluation
can be used to revise
instruction as the course is being developed and
implemented. For example, the distance educa-
tor can give students pre-addressed and stamped
postcards to complete and mail after each session.
These “mini-evaluations” might focus on course
strengths and weaknesses, technical or delivery
concerns, and content areas in need of further
coverage.
Within the context of formative and summative
evaluation, data are collected through quantitative
and qualitative methods.
Quantitative evaluation
relies on a breadth of response and is patterned
after experimental research focused on the col-
lection and manipulation of statistically relevant
quantities of data.
In contrast,
qualitative evaluation
focuses on a
depth of response, using more subjective methods
such as interviews and observation to query a
smaller number of respondents in greater depth.
Qualitative approaches may be of special value
because the diversity of distant learners may defy
relevant statistical stratification and analysis. The
best approach often combines quantitative mea-
surement of student performance with open-ended
interviewing and non-participant observation to
collect and assess information about attitudes
The benefits of evaluation in engineering e-
Learning are (Astrom, 2008):
•
Justify and expand training funds.
•
Selecting the best instruction strategies.
•
Obtain and analyze learners' fulfillment
feedback.
•
Quantify performance improvement.
And the risks of not evaluating (Astrom, 2008):
•
Waste money on ineffective programs.
•
Terminate training program.
•
Ineffective and Inefficient transfer of training.
•
Training program does not support
objectives.
•
Learners' training needs not met or
unknown.
•
Presentation improvement unknown.
•
Expectations are not set nor communicated.
Search WWH ::
Custom Search