How We Know What Isn't So (26 page)

I
n the mid-1940s, a surgical procedure known as the portacaval shunt was developed to treat esophogal hemorrhaging, and its use was later expanded to treat a variety of ailments of the intestinal cavity. Studies of the effectiveness of this procedure were carried out in numerous hospitals, and, twenty years after the development of the surgery, the literature on this subject was reviewed.
¹

Table 10.1
presents the results of this review. Each study was categorized along two dimensions: the methodological quality of the study (i.e., degree of experimental control) and the amount of enthusiasm the investigators expressed about the procedure as a result of their study. As
Table 10.1
makes clear, the investigators’ enthusiasm for the procedure was negatively correlated with the quality of the study. Uncontrolled and poorly-controlled studies produced considerable enthusiasm for the shunt; well-controlled studies generated virtually none.

This investigation teaches several important lessons. First, most research in this area (and in many others as well) is of poor quality. Only 4 of the 51 studies employed the absolutely essential procedure of randomly assigning participants to treatment conditions. The quality of research has undoubtedly improved since the time these studies were carried out, but badly-flawed research has by no means disappeared. Second, as already mentioned, the conclusions drawn often depend on the quality of the research performed.

Table 10.1
Degree of Experimental Control versus Degree of Investigator Enthusiasm for the Portacaval Shunt
Degree of Enthusiasm

Faulty research designs can obscure phenomena that are really there, or lead us to believe in phenomena that are not. Finally, and most importantly for present purposes, the existence of a large number of studies does not by itself compensate for their lack of quality. Adding together a set of similarly-flawed investigations does not produce an accurate assessment of reality. As statisticians like to say, sample size does not overcome sample bias.

This last lesson is sometimes hard for people to accept. Somehow it just seems that if one conducts enough studies, the flaws should cancel each other out and allow “the truth” to shine through. Like Woody Allen“s Catskills vacationer, people tend to think that sufficient quantity can compensate for a lack of quality. There are domains in life in which it does, but as the example of the portacaval shunt makes clear, empirical research is not one of them.
*

This lesson is important to keep in mind when trying to determine why so many people believe in the existence of extra-sensory perception (ESP). One could argue that the single most important reason is simply that there seems—to the average person—to be so much apparent evidence for it. Everyday life brings its share of support as friends tell us about premonitions that have come true, and we experience extraordinary coincidences ourselves. Press reports provide us with even more apparent evidence: Scientists at “Stanford” test Uri Geller under controlled conditions and conclude that his psychic powers are genuine.* Reports from the Soviet Union suggest that the Russians are ahead of us in psychic espionage and warfare. Psychics routinely predict upcoming events and help the police solve criminal cases. Celebrities tell of their prophetic dreams or their memories of “past lives.” The list goes on and on. As we shall see, none of this apparent evidence provides truly solid support for the existence of ESP, but the examples do pile up.

Some people accept these examples at face value and conclude that ESP is a fact of life. Others may question these claims, or may be vaguely aware that some of them have been scientifically challenged, but nevertheless decide that “there must be something there.” Much of the evidence may be fraudulent or faulty, but there is so much of it. Can’t we conclude that “where there’s smoke, there must be fire?” We cannot, of course, as the example of the portacaval shunt makes clear. But this reasoning is seductive nonetheless. Indeed, it is not just the average person who falls prey to this fallacy. Well-trained scientists have been known to make the same argument. Stanford professor William Tiller, for example, has argued that even though the evidence from ESP experiments is shaky, it should be taken seriously because there is so much of it.
*
Similarly, parapsychologist John Beloff has stated:

Sure the food is terrible, but such large portions! It seems clear from all of this that one of the most powerful determinants of people’s dubious belief in ESP is simply the availability of so much apparent evidence of its existence—evidence from both everyday life and the laboratory. What does this evidence really show?

Is it fair to characterize belief in ESP as dubious? Perhaps some readers will feel that I have jumped the gun by making such a, well, dubious statement. Is ESP as questionable as I seem to imply? To address this question requires a brief critique of the laboratory evidence offered in support of ESP.
*
First, however, it is important to clarify the precise meaning of the concepts and phenomena under discussion. Extrasensory perception is defined by those who study it (known as parapsychologists) as the “experience of, or response to, a target object, state, event or influence without sensory contact.”
⁴
Several types of ESP are thought to exist.
Telepathy
refers to the direct transfer of thoughts from one mind to another;
clairvoyance
corresponds to the ability to sense or “see” events and objects that are absent from one’s visual field; and
precognition
represents the perception of future events. A fourth ability,
psychokinesis
(
PK
), or the ability to move, alter, or influence objects without any known material contact, is also generally considered to be a component of ESP because it too represents influence at a distance. However, because psychokinesis does not involve “perception” in any sense, parapsychologists now generally include it with the other abilities under the more general term “psi.”

It may be best to begin an examination of whether these phenomena are likely to exist by looking at a sample of opinion from people with strong ties to this field. A scientific panel commissioned by the National Research Council to study this area concluded that “… despite a 130-year record of scientific research on such matters, our committee could find no scientific justification for the existence of phenomena such as extrasensory perception, mental telepathy, or ‘mind over matter’ exercises…. Evaluation of a large body of the best available evidence simply does not support the contention that these phenomena exist.”
⁵
Ray Hyman, a psychologist who has devoted much of his career to evaluating claims of paranormal phenomena, similarly states that “… there is no scientifically acceptable basis, as of today, for accepting the reality of psi.”
⁶
Even many of those who fervently believe in the reality of psi can sound a similar theme. Stanley Krippner, a firm believer in psi and an articulate advocate for parapsychology, nevertheless states that “since Charles Richet first applied statistics to psychical research data nearly 100 years ago, no experimental procedure has emerged which would invariably produce the same results no matter who followed it. Furthermore, no mechanism underlying psi has been discovered … Finally, no practical use of ESP or PK has been validated by laboratory research.”
⁷

Hardly a ringing endorsement. In part, this gloomy assessment of the status of ESP stems from a disturbing pattern that has repeated itself over the past 130 years. First, the believers and skeptics stake out their positions, the believers by citing anecdotes of unexplained phenomena in everyday life, and the skeptics by noting the inherent implausibility of psi (e.g., its existence would violate a number of physical laws such as the inverse square law and the second law of thermodynamics). While the debate rages on, the parapsychologists energetically conduct experiments on psi, and, at some point, produce supposedly “definitive” evidence. At first blush, the evidence can seem rather convincing and the initial skeptical response can sound rather weak and even petty. Convinced that they hold the upper hand, the believers then chide the skeptics for their closed-mindedness. The skeptics are likened to the medieval clerics who refused to look through Galileo’s telescope and persecuted those who espoused the heliocentric view of the solar system. They are castigated as representatives of a scientific “establishment” who stand in the way of unprecedented progress in our understanding of our world and ourselves.

The believers’ euphoria does not last long, however. As soon as enough time has elapsed to allow sufficient scrutiny of the evidence, it generally becomes clear that it is hardly definitive. Rather, it is often shown to be the result of deliberate fraud or critical methodological shortcomings. What is once offered as the very best evidence for ESP becomes an embarassment to the field. Consider a few examples from the history of parapsychological research during the past 130 years.

J. B. Rhine.
J. B. Rhine is generally credited with initiating the investigation of psi under laboratory conditions. Before Rhine, research on psychic phenomena, carried out under the auspices of the Society for Psychical Research in Great Britain and the American Society for Psychical Research in the U.S., focused mainly on the investigation of spontaneous experiences of psi and on the study of spiritual mediums who claimed to be able to contact the souls of the dead. (Note that the niche occupied by the mediums of the early twentieth century is now filled by modern-day “channelers.”) At his Parapsychology Laboratory at Duke University in the 1930s, Rhine developed an experimental procedure in which subjects were asked to identify which of five symbols (circle, cross, rectangle, star, or wavy lines) were on the concealed sides of a deck of cards. The number of correct responses was then compared to the number to be expected if the subject was simply guessing. This procedure can be used as a test of telepathy (a “sender” looks at each card while the “percipient” tries to discern its identity), clairvoyance (the card is set face down on the table without anyone having seen it), or precognition (the percipient guesses the sequence of cards from top to bottom, and
then
they are shuffled and compared to the percipient’s responses).

In 1934 Rhine published the results of several years of research involving nearly 100,000 guesses made by a large number of subjects. Rhine concluded that he had obtained overwhelming evidence for the existence of ESP. His subjects averaged 7.1 correct identifications per deck of 25 cards compared to the 5 expected by chance. Although the magnitude of the results may be fairly small (i.e., only two extra successes per deck), the odds of such a result happening by chance over such a huge number of trials are virtually zero.
⁸

Rhine’s results quickly reached a wide popular and scientific audience. Because the research had the veneer of carefully-controlled scientific research, the skeptical community initially was at a loss about what to think. Gifted subjects performed undeniably better than one could do by chance. However, as more details about the exact procedures employed became known, the data began to lose their lustre. Most observers became convinced that the results were due to methodological artifact rather than genuine ESP. Some of Rhine’s subjects, for example, were allowed to shuffle and handle the cards, a procedure that would permit a conjurer to use trickery to discern their identity. One of Rhine’s most gifted subjects, Hubert Pearce, was consistently able to obtain positive results under such conditions—except when persons other than the experimenter were present.
⁹
Skeptics find this pattern—of successful performance when cheating is relatively easy, and unsuccessful performance when it is difficult—to be extremely suspicious. Believers, in contrast, attribute it to an “experimenter” or “shyness” effect whereby a person’s psi powers disappear in a skeptical atmosphere. Psychics, like temperamental Rock stars, often do not perform in the presence of bad vibes.