Evaluating Telepresence Robots in the Field - Agents and Artificial Intelligence - page 430

progressive work toward a long-term human-robot interaction assessment and then we
discuss some lessons learned from a more focused technological point of view.

3 The Evaluation Effort

We have conceived a twofold path for evaluating the human-robot interaction gathering
both feedback from short interactions between potential users and the G IRAFF robot
and also focusing on a long term assessment plan. More specifically we identified two
tracks for our effort:
- Short Term Evaluation , which consists of a collection of immediate users feedback
(i.e., after a short interaction with the robot) on the telepresence robot, connected
to different aspects of the interaction mainly related to the usability, willingness to
adopt it, possible domains of applications, advantages and disadvantages.
- Long Term Evaluation , which relates to the study of the long-term impact of G I -
RAFF 's social and physical presence on elderly users using the system both to
communicate with their relatives and friends and to receive visits from healthcare
providers and in general caregivers.
The short term evaluation effort, though not sufficient alone, still provides immediate
feedback that can be used to quickly improve the technological development, to possi-
bly add functionalities to the system or to simply confirm the validity of some techno-
logical choices. In addition it can give valuable guidance to the long-term assessment.
For this reason we adopted a combined approach involving participants representative
of both types of users: the client side and the end user side .
Following this schema, we first present results for the short term evaluation per-
formed in Italy, then we introduce our complete design for a methodology to assess
the long-term impact also reporting the status of the Italian long-term test sites that are
currently running according to this methodology.

3.1 Short Term Evaluation
As stated in [7], before intelligent technologies would be accepted, it is important to un-
derstand their perception of the benefits, concern and adoption criteria. In our study, we
aim at reproducing as much as possible an “ecological” setting for the experiment. To
this purpose we distinguished the role of the users and recruited different participants
according to their expected role. Specifically for the client side, we recruited users rep-
resentative of the potential visitors of the elderly users among caregivers, nurses, health
workers, etc. For the end user side we interviewed older adults living alone, or possibly
receiving some kind of health care assistance. In the following we describe the general
method and the material and procedure used for both types of users.

Method. This evaluation was aimed at assessing users reaction toward the possible
adoption of the G IRAFF system as a means to visit or provide some kind of service to
the elderly users. Aspects investigated were willingness to adopt the robotic solution ,
possible domains of application , advantages and disadvantages and suggestions for
improvements .