Information Technology Reference
In-Depth Information
The results show that, looking at the numbers and percentages, no system outper-
forms the others, which on the other hand confirms our approach. Please note that the
TEP
approach works for query-driven context-oriented named entity recognition only.
This means that all approaches used in this evaluation clearly have their benefits in other
application areas.
Nevertheless by going into details we saw some remarkable differences between the
results the systems produced. All systems were able to extract the main general NEs like
locations or persons. For terms that are important in the context of actuality and current
developments, we saw that the
TEP
approach is able to extract more relevant items.
In case of “Fukushima”, the
SProUT
system did not extract terms like “eartquake”,
“tsunami” or “nuclear power plant”. Of course this is because the underlying ruleset
has not been developed for covering such types of terms. The
AlchemyAPI
and
Stan-
fordNER
systems were able to extract these terms but failed in detecting terms like “ac-
cident” or“safety issues”. For “Justin Bieber” relevant items like “movie”, “tourdates”
or “girlfriend” could not be detected by all systems except
TEP
. For the snippets asso-
ciated with the query “New York” all systems identified the most important NEs, and
differed for less important NEs only.
Last but not least the runtime, which plays an important role in our system, varied
from 0.5 seconds for the
SProUT
system, to 2 seconds for
TEP
, 4 seconds for
Stanford-
NER
to 15 seconds for
AlchemyAPI
.
6
Evaluation of the Touchable User Interface
For information about the user experience we had 26 testers — 20 for testing the iPad
App and 6 for testing the iPhone App: 8 came from our lab and 18 from non-computer
science related fields. 15 persons had never used an iPad before, 4 persons have been
unfamiliar with smartphones. More than 80 searches have been made with our system
and with Google respectively.
After a brief introduction to our system (and the mobile devices), the testers were
asked to perform three different searches (using our system on the iPad, iPhone and
Google on the iPad/iPhone) by choosing the queries from a set of ten themes. The
queries covered definition questions like
EEUU
and
NLF
, questions about persons like
Justin Bieber
,
David Beckham
,
Pete B est
,
Clark Kent
,and
Wendy Carlos
, and general
themes like
Brisbane
,
Balancity
,and
Adidas
. The task was not only to get answers
on questions like “Who is
...
”or“Whatis
...
” but also to acquire knowledge about
background facts, news, rumors (gossip) and more interesting facts that come into mind
during the search.
Half of the iPad-testers were asked to first use Google and then our system in order
to compare the results and the usage on the mobile device. We hoped to get feedback
concerning the usability of our approach compared to the well known internet search
paradigm. The second half of the iPad-testers used only our system. Here our research
focus was to get information on user satisfaction of the search results. The iPhone-
testers always used Google and our system mainly because they were fewer people.
