Information Technology Reference
In-Depth Information
Table 2.
Questionnaire result of the experiment using a portable translation system
Questionnaire items
Average Standard
deviation
(1) The translation result of my chat sentence was correct.
2.2
0.16
(2) I think that the back translation is necessary.
4.6
0.24
(3) I understood the meaning of the foreigner's (the other party
of the chat) sentence from the translation.
2.8
0.56
(4) I am satisfied with the accuracy of the translation.
2.0
0.40
(5) I understood the usage of the translation function at once.
4.0
0.40
(6) A problem occurred in the translation system while chatting.
2.2
1.36
(7) I think that the machine translation in the chat is useful.
3.6
1.04
We used a five-point Likert scale for the evaluation, with 1 = Strongly disagree, 2 = Disagree, 3 =
Neutral, 4 = Agree, and 5 = Strongly agree.
2. The subject went to an island (called SIM in Second Life) where the other
avatar existed.
3. The subject read a conversation among the general users on Second Life and
chatted to them.
The duration of each experiment was approximately 20 or 30 min.
In the experiment, the translation system was not able to translate sentences
in many cases. Many users in Second Life used clipped forms for chat. Some users
incorrectly used capital and small letters while spelling out a word. Moreover,
even if the chat with a general user on Second Life was translated, it was not
possible to follow the conversation because the translation could not keep up
with the speed of the conversation. Subjects were ignored by almost everyone
on Second Life.
Table 2 shows the questionnaire result of the experiment using the portable
translation system.
We obtained good results with respect to the necessity and ease-of-use of the
system from the questionnaire (Table 2 (2), (5), and (7)). However, the results
with respect to the actual use of the system in Second Life were poor (Table 2
(1), (3), and (4)). We found that the sentences of the other party and those of
the subject were often translated incorrectly. The main reasons that the subjects
gave for not being able to carry out a good chat conversation (Table 2 (3)) were
as follows:
-
I hardly understood the content of the chat. However, I think that the trans-
lation depends on the content of the sentence.
-
I think a peculiar word was often used in the chat ('You' is shown 'u'). I
did not know what was said at all because the contents of the chat were too
unusual.
-
Because the translation results were not too correct, I could not understand
the conversation well.
