The statistical methods we have been discussing permit you to estimate the certainty of statements and precision of measurements that are common in the biomedical sciences and clinical practice about a population after observing a random sample of its members. To use statistical procedures correctly one needs to use a procedure that is appropriate for the study design and the scale (i.e., interval, nominal, ordinal or survival) used to record the data. All these procedures have, at their base, the assumption that the samples were selected at random from the populations of interest. If the study as conducted does not satisfy this randomization assumption, the resulting P values and confidence intervals are meaningless.
In addition to seeing that the individuals in the sample are selected at random, there is often a question of exactly what actual populations the people in any given study represent. This question is especially important and often difficult to answer when the experimental subjects are patients in academic medical centers, a group of people hardly typical of the population as a whole. Even so, identifying the population in question is the crucial step in deciding the broader applicability of the findings of any study.
Taking all the information we have discussed on cell phones and sperm allows us to confidently conclude that exposure to cell phones adversely affects sperm. We began in Chapter 3 with two human observational studies showing lower sperm motility. The first one* showed a difference between men with lower and higher cell phone use. The second study† improved upon this design by including a true control group of men that did not use cell phones at all as well as including several levels of use and finding a dose–response relationship, with greater reductions in sperm motility associated with increased levels of cell phone use. These two studies, however, were observational, leaving open the possibility that the relationships they elucidated were actually reflecting the effects of some unobserved confounding variable. Concern over confounding variables is especially acute because all the men providing the sperm samples were recruited at fertility clinics, so, even though the investigators tried to screen out men with other reasons for reproductive problems, the possibility remained that something else than exposure to cell phone radiation was causing the reduction in sperm motility.
We increased our confidence that the cell phone radiation was actually affecting sperm when we considered an animal experimental study‡ that showed that rabbits exposed to cell phone radiation had depressed sperm motility. Unlike the earlier two human studies, these results came from an experiment in which the rabbits were randomized to the different treatments and in which the investigators controlled the environment, so we can be much more confident that the results were the result of the cell phone radiation causing the observed changes rather than them being a reflection of some unobserved confounding variable. The issue of interspecies extrapolation, however, remains.