Information Technology Reference
In-Depth Information
that the list of relevant services should be automatically updated in time when enter-
ing new requirements.
However, results from the debriefing sessions indicated that most usability defi-
ciencies were not critical to the main tasks and that the current S
3
C Solution Explorer
prototype does fulfill the consultants' key requirements. With these results, we
decided to use the S
3
C Solution Explorer without further development in more natu-
ralistic studies in order to explore its utility in more detail.
4.2 S
3
C Solution Explorer Utility Evaluation at InsideAx
This study investigated the effect of the S
3
C Solution Explorer on the discovery of
relevant services. We explored whether consultants need less time to identify relevant
services with the help of the S
3
C Solution Explorer compared to non-tool supported
service identification. Furthermore, we investigated the correctness and completeness
of these solutions. Three consultants from InsideAx participated in this S
3
C Solution
Explorer evaluation. One has a master degree in business informatics and more than
three years of experience in ERP system customization (Consultant 1). The second
consultant had more than three years of experience in ERP system customization
while the 3rd consultant only had one year experience. None of the consultants is an
author of this paper, and none had had contact with the S
3
C Solution Explorer prior to
the evaluation.
As for the first study, each evaluation was structured in briefing, evaluation and
debriefing. For the evaluation we prepared an application example which included
three use cases and typical use case requirements derived from previous projects.
For each of the use case we reviewed existing implementations and ERP system docu-
mentations and defined a list of relevant services out of more than 70 available ser-
vices. This standard solution was used as a basis for evaluating consultants' solutions.
The application example consisted of following use cases:
•
Item Arrival
describing the handling of delivered goods from vendors. Five rele-
vant services were identified as standard solution for this use case.
•
Transfer production goods to warehouse
representing events relevant for inven-
tory management. Again, five relevant services were identified as standard
solution for this use case.
•
Buy Item in a Web Shop
discussing sale related events. Six services were consid-
ered to be the standard solution for this use case.
In the briefing session, the consultants were told about the purpose of the study. We
further discussed the application example with the consultants. Each of them was
asked to estimate the time he would need for the identification of relevant services
without tool support (see Table 1). During the evaluation each consultant used the
S
3
C Solution Explorer to identify relevant services for the three use cases and to doc-
ument the results. The debriefing session was used to discuss their solution and to
identify utility issues regarding the S
3
C Solution Explorer. We compared particularly
the time required for the identification of relevant services to the estimates. Further-
more, the developed solutions were compared to the standard solution in terms of
correctness and completeness (see Figure 7).
