Database Reference
In-Depth Information
“Yes, but some task completions were about the same. But we deinitely have our
work cut out for us if we're going to implement the new search.”
“You got that right, mister research,” he says, lipping through the pages. “I gotta
tell Joey about this. He's gonna blow a gasket.” He takes another sip. “But tell me
something. On task 1, the Palo Alto guys got 3 completions and we got 9. But you only
have 10 participants for each engine. How does this translate to a larger audience?”
“Great question. I can calculate the conidence intervals for the pass rates for you;
perhaps they will give you more insight about the results.”
“Cool. Get me that asap, so I can really deliver the full story to Joey. It's still
gonna be a nightmare, but I wanna have our story straight. I don't want to invade if
there's no weapons of mass destruction; follow?”
“You chuckle at the Iraq war reference.” “Yeah, I got it. I'll get you some num-
bers by the end of business today.”
4.5
BINOMIAL CONFIDENCE INTERVALS AND THE ADJUSTED
WALD METHOD
There are a variety of ways you can go about forming a conidence interval for the
true “pass proportion,” based on a small sample for data. We will focus on what
might be called “binomial conidence intervals.”
SIDEBAR: BINOMIALLY NORMAL?
In a way, “binomial conidence interval” is an appropriate name, while in another way, it isn't. The
underlying data-generating process is a binomial one. Each data point is a “pass/fail” (having two and
only two outcomes is the core of the term “binomial”; the preix, “bi,” corresponds with this). There are
also other assumptions that go along with having a so-called binomial process. We describe them a bit
later. In this sense, the name is appropriate. On the other hand, the method that is considered the most
appropriate for determining these conidence intervals does not actually involve using probabilities from
a binomial table, but, rather, uses values from a normal distribution table, as we shall describe. This is
another example of just how prevalent and useful the normal curve is, and why we spent the time we did
in Chapter 1 on the normal curve. Still, it suggests that the name “binomial conidence intervals” is not
directly appropriate, even though we are inding a conidence interval for a true proportion.
The method that is considered the most appropriate for inding a conidence inter-
val for a proportion (e.g., the proportion that will use a search engine successfully),
when the underlying process is considered binomial (i.e., pass/fail) and sample sizes
are small, is the “adjusted Wald method” (
Agressi & Coull, 1998; Sauro & Lewis,
2005; Lewis & Sauro, 2006
). The theoretical assumptions to have a binomial process
are that we have two and only two outcomes to a “trial” (here, successfully complet-
ing a task or not), that we have independence between different outcomes (here, in
essence, each data point for a given task corresponds with a different person), and
last, that the true probability of “pass” (here, the proportion who would pass if we
considered the entire population of people who would ever undertake the task, which
is, of course, unknown) is a constant for each person, given no information about the
person. Since the last assumption may not be strictly true, we are “approximating” a
binomial process, but it is more than adequate for our purposes.
Search WWH ::
Custom Search