Information Technology Reference
In-Depth Information
4.1 S
3
C Solution Explorer Initial Utility and Usability Evaluation at Terna
Three consultants participated in the initial utility evaluation of the S
3
C Solution Ex-
plorer at Terna. Each has a master degree in business informatics and more than three
years of experience in ERP system customization. None of the consultants is an
author of this paper, and none had had contact with the S
3
C Solution Explorer prior to
the evaluation. However, two of the consultants also participated in the first interview
where requirements for S
3
C approach were identified (see Section 3.1).
Each evaluation was structured in 3 parts - briefing, evaluation and debriefing.
During the briefing each participant was informed of the study's purpose and the task
they were intended to perform. The briefing also included a short introduction to the
S
3
C Solution Explorer. For the study we prepared a typical ERP use case describing
how a warehouse worker handles the receipt of goods - the item arrival use case. The
example use case also included four requirements which were gathered from custom-
ers in previous projects (e.g.
The system should provide a unique identification num-
ber for each vendor
). During the evaluation each consultant used the S
3
C Solution
Explorer to enter the use case and the related requirements. Furthermore, they were
asked to use the Solution Explorer to identify relevant services for that use case. One
of the authors observed each consultant who spoke loudly throughout the process. The
utility of the S
3
C Solution Explorer concerning the previously discovered consultants'
requirements (see Section 3.1) was discussed during the debriefing. The debriefing
was then used to discuss usability problems of the S
3
C Solution Explorer. All 3 con-
sultants completed the evaluation and debriefing, which lasted on average 20 and 25
minutes respectively.
Observations and qualitative feedback were encouraging but they also identified
some limitations. The study revealed that the S
3
C Solution Explorer does fulfill most
of the consultants' requirements (see Section 3.1):
•
The consultants were able to document use cases and requirements.
•
The consultants said that the tool provides a clear structure in linking use cases
and requirements.
•
The consultants were able to use the tool to run service queries.
•
The consultants said that the tool suggested relevant service solutions.
•
The consultants said that a ranked list of relevant services and the provided
matching probabilities supported them in narrowing down the discovered results.
•
The consultants said that the current service description does provide key infor-
mation which (in most cases) allows them to decide whether a service is relevant.
However, they requested more detailed service descriptions to improve the deci-
sion process. For example, consultants requested information about the usage of
the service in previous projects.
In the debriefing meetings, the consultants also highlighted the fact that they would be
willing to use a tool such as the S
3
C Solution Explorer for their daily work with cus-
tomers. However, the interview also revealed several usability issues. They pointed
out that the first prototype does present too much information at a time and that it is
therefore hard to keep an overview. They argued that service discovery queries take
too much time (on the average 3 seconds) and that they would prefer quicker re-
sponses in order to strengthen seamless discussions with customers. They requested
