Information Technology Reference
In-Depth Information
There are other textbooks such as Converse and Presser [1986], Fowler
and Fowler [1995], and Schroeder [1987] to assist you in wording ques-
tionnaires and in pretesting questions for ambiguity before you begin. We
have only two paragraphs of caution to offer:
1. Be sure your questions don't reveal the purpose of your study;
otherwise, respondents shape their answers to what they perceive
to be your needs. Contrast “how do you feel about compulsory
pregnancy?” with “how do you feel about abortions?”
2. With populations ever more heterogeneous, questions that work
with some ethnic groups may repulse others (see, for example,
Choi [2000]).
Recommended are web-based surveys with initial solicitation by mail
(letter or post card) and email. Not only are both costs and time to com-
pletion cut dramatically, but also the proportion of missing data and
incomplete forms is substantially reduced. Moreover, web-based surveys
are easier to monitor, and forms may be modified on the fly. Web-based
entry also offers the possibility of displaying the individual's prior
responses during follow-up surveys.
Three other precautions can help ensure the success of your survey:
1. Award premiums only for fully completed forms.
2. Continuously tabulate and monitor submissions; don't wait to be
surprised.
3. A quarterly newsletter sent to participants will substantially
increase retention (and help you keep track of address changes).
DETERMINING SAMPLE SIZE
Determining optimal sample size is simplicity itself once we specify all of
the following:
•
Desired power and significance level.
•
Distributions of the observables.
•
Statistical test(s) that will be employed.
•
Anticipated losses due to nonresponders, noncompliant partici-
pants, and dropouts.
Power and Significance Level
Understand the relationships among sample size, significance level, power,
and precision of the measuring instruments.
Sample size must be determined for each experiment; there is no uni-
versally correct value (Table 3.1). Increase the precision (and hold all
other parameters fixed) and we can decrease the required number of
observations.
