Cross-Cultural Design and Evaluation Framework - Cross-Cultural Human-Computer Interaction and User Experience Design - page 83

7 Cross-Cultural Design and

Evaluation Framework

7.1 RESEARCH METHODS
To compare the HCI/UX across different cultures we evaluated the following options:

• Run a usability testing between Adobe Photoshop and the GIMP. The results
would enhance the expert evaluation done in Part I, “Semiotics of Interac-
tion.” On the other hand, some of the users may have been familiar with the
applications, which would distort the results. The test setting would be quite
complex, and the results may have not shown a very large scope of cultural
markers, because both the applications come from the Western context.
• Run a usability testing of webpages from different cultural backgrounds. In
this test we would gather more diverse results. However, some of the pages
share the same design patterns across cultures, some of them are localized
versions of Western web portals, which would limit our research scope.
• Run a usability testing of a UI prototype built (or modified) according to
our proposed guidelines. This would be perhaps the most time-intensive and
risky approach to the research. Risky, because we would build on top of
guidelines that we did not evaluate before.
• Conduct an ethnographic observation and user interview. This kind of re-
search would gather a large scope of cultural insights, but not all of them
would be comparable between cultures. Also, we would not be able to focus
on many cultural markers.
• Validate both previous research results and newly generated hypotheses
through usability testing and interviews. In this way we would create a
ground, from where we could build our guidelines. Validating the results
and hypotheses does not allow delving very deep into different research
subjects, but would allow us to cover a large ground, while working with a
larger sample. We chose this kind of method for our pilot study.

In order to find the prevalent and preferred UI language components and cultural
markers, we focused our study on the five following areas: personal information (de-
mographics, exposure to other cultures and technologies), layout (discrete elements,
patterns, interaction sentences and narration), color (discrete elements, rhetorical
tropes), symbol (rhetorical tropes), and look and feel (interaction sentences, narra-
tion, patterns, and rhetorical tropes). There were few overlaps due to the broad scope
we focused on in this pilot study.
Our qualitative method was based on one-to-one and one-to-many interviews sup-
ported by note taking and filling in questionnaires. In order to test the different hy-
potheses, we created at least one question for each hypothesis. The questions were

83