Information Systems Outsourcing - Enterprise Information Systems Design, Implementation and Management: Organizational Applications - page 416

• identify success key factors for the on-

line Information Services of Science and
Technology Management Systems, by
Santos, 2004;

reached one can say the listed issues ranking is
reliable (Amaral et al., 2003). The concept of a
group consensus is the opinion homogeneity or
consistency among the participants. In previous
studies the Delphi round process has been stopped
by the lack of time to proceed or the decrease of
answer rates between a round to another (Santos,
2004). In this study the Kendall coefficient of
agreement W was adopted as a way to evaluate
the level of consensus among the experts, since it
has been widely applied with the Delphi method in
different knowledge areas to identify and prioritize
key issues (Miguel, 2000). This Kendall's W verify
the agreement among the experts, measuring the
correlation of the ranked issues (Miguel, 2000).
The coefficient value increase with the level of
consensus, varying from 0 (without consensus) to
1 (perfect consensus) (Schmidt, 1997). This coef-
ficient cannot be negative because when several
experts are involved they can completely agree
but they will not entirely disagree (Miguel, 2000).
When Kendall's coefficient is small the deci-
sion is trivial but when consensus is moderate it
is difficult to come to a decision (Schmidt, 1997).
Schmidt (1997) points out two criteria that can be
used to stop the round process of a Delphi study:

• give more structure to the field of

Knowledge Management and to get an
outlook on developments for the next ten
years, by Scholl et al., 2004;

• identify key issues of Information Systems

Management in the Portuguese environ-
ment, by Campos, 1998;

• present a systemic framework of what the

field of Information Systems is about, by
Bacon & Fitzgerald, 2001;

• obtain Information Systems managers'

view about Outsourcing in Spain, Gonzalez
et al., 2006a.

It is rather usual to combine the Delphi method
with sorting techniques as the Likert or the Q-sort
scales.
The Likert scale is the most frequently used
technique in key IS issues research, therefore
became the most well-known and studied scale
(Campos, 1998). In this classification, the expert
judges each issue independently giving it a scale
value and does not consider the issue as a part
of a whole. This way, turns difficult to judge the
importance of each issue relatively to the other
issues, and it is common to give the extreme scale
values, which contributes to obtain different issues
with the same classification.
In this case the Delphi study was conducted
with the Q-sort technique because it would serve
the purpose of obtaining risks and benefits of
Outsourcing ranked according to their importance
(Okoli & Pawlowski, 2004). With this technique
participants are asked to rank instead to rate and
to take in consideration all issues simultaneously
(Amaral et al., 2003) which prevents equal posi-
tions among listed issues.
The Delphi should be conducted through as
many rounds as the necessary to achieve a rea-
sonable level of consensus. When consensus is

• In case of strong consensus (W>=0.7) or

• In case of absence of strong consen-

sus among the experts, a levelling off of
Kendall's W from one round to the next,
which indicates a lack of progress between
rounds.

Schmidt (1997) refers that for a group of more
than 10 experts, even very small values of W can
be significant.

eMpiricAL studY MethodoLogY

The study methodology was a qualitative survey
carried out in Portugal, using the Delphi method
with Q-sort, to obtain the IS market players per-