Information Technology Reference
In-Depth Information
The computation of linear regression returns a measure of error, but what this value
means in practice is difficult to interpret. However, the visualization of the fit is
striking: the line rides neatly between the points.
The graphs in Fig.
15.3
compare the ability of two systems to respond to 50 events
(the score is a human-assigned value for quality of response; again, this is data from
a real experiment). In the upper graph, System 1, with the crosses, often appears to
be better than System 2, with the triangles; but in a reasonable number of cases the
reverse is observed.
Which is better? Wilcoxon's signed rank hypothesis test reports that, for a spec-
ified level of 99 % confidence, System 1 is superior. This can be confirmed through
visualization. One possible visualization is shown in the lower graph in Fig.
15.3
,
where the events have been sorted by the performance on System 1. The crosses now
form a clear line; while a few of the triangles are above, the majority are below. It is
a simple transformation, but highly informative. The data is simply a set of matched
pairs: there is no innate sequence, and so the pairs can be reordered.
Another example of visualization is shown in Fig.
15.4
. In the upper figure, a dot
plot has been used to capture the relationship between the effectiveness of a baseline
query evaluation technique and the improvement available through an alternative
method. The hypothesis was that queries that were originally successful would be
less amenable to further improvement than queries that were originally poor. Orig-
inal effectiveness and new effectiveness are strongly correlated: a query that can be
resolved with one method can also be resolved with the other. However, as the figure
illustrates, there is no clear indication that poor queries can be improved more than
others.
An alternative view is presented in the lower figure, where the effectiveness values
on the horizontal axis have been averaged across subranges of width 0.05. This graph
shows that the improvements are more or less the same, independent of the original
effectiveness of each query, and thus suggests that there is no correlation.
Tools for visualization of data continue to develop, with rich mechanisms that
allow dynamic interrogation and reinterpretation of the underlying behaviour. How-
ever, even elementary visualization can be extraordinarily revealing. Such analysis
is often the best way to explore and explain data.
A “Statistical Principles” Checklist
•
What variables might influence your results? Will analysis of these variables mean
that you need to make use of statistics?
•
Can you predict the effect of altering each variable? How do they interact? Are
they independent?
•
How do the experiments distinguish between the effects of the variables?
•
Are effects random or systematic? How are they to be controlled?
•
What method will be used to investigate outliers?
