Information Technology Reference
In-Depth Information
7.6.2.
Final evaluation by professional drivers
The intermediate evaluation was immediately followed by a new phase of
specification and development in order to update the HMIs and the assistance
strategies for the final evaluation stage.
This final stage of the VIVRE2 project aimed to evaluate the relevance,
functioning principles and strategies of the system as well as that of their restitution
mode (visual interfaces and alarms). From an ergonomic point of view, the VIVRE2
system was evaluated by drivers in terms of usability and acceptability, perceived
utility and efficiency as well as understanding of assistance strategies. The mental
work-load caused by use of the system in the active and informative mode was
measured for each subject, and compared to a driving situation with no system.
Fifteen male professional drivers (average age: 45) participated in the trials. For
each of them, the experimental procedure consisted of four sessions lasting one hour
each and spread over four non-consecutive days. These four sessions were carried
out on a complex and dense urban circuit and enabled three conditions to be tested: a
“familiarization” condition, a condition with “informative system only” and a
condition with “informative and active system”. Each condition consisted of a
different itinerary, simulating a delivery round in a town with different maneuvers to
carry out in order
to be able to park. Six test situations were reproduced in each
condition to enable the subjects to evaluate the system in different cases of use and
to test its efficiency by counting the number of accidents that are “potentially”
avoided. In order to control the impact of learning, the itineraries and test situations
were presented to subjects randomly. Moreover, the situations were positioned in
different places on the circuit for each experimental condition. Finally, each
situation was replicated at least twice per condition with and without criticality and
presented in a random way. For example, reversing to park in a pedestrian zone or a
cul-de-sac, condition 1 was carried out without a vulnerable user behind the truck
and condition 2 had one or two pedestrians having a discussion behind the truck.
Each session was made up of a “moving forward” part on the simulator followed by
an “ergonomic evaluation” part. The mental work-load of drivers was evaluated with
the help of a modified version of the NASA TLX
7
[HAR 88]. This method was
adapted for the situations of evaluation of assistance systems for driving a truck on a
simulator [MAI 11]. The dimensions taken into account in this version are attention,
visual requirements, auditory requirements, disturbance (here caused by the
7 The NASA Task Load index is a multidimensional method destined for subjective
evaluation of the workload felt by an operator. The first three dimensions represent the
damage imposed on the subject by the task (mental, physical and time requirements) and the
three others account for the interactions of the subject with the task (performance, effort and
frustration). The test provides a global weighted score of the workload and enables
identification of the specific task for a given task or system.
