of Draft January 18, 2004
Page 1
Chapter 1b. PSYCHOLOGICAL SCIENCE AS A WAY OF KNOWING: A Case Study
One of the best examples of using scientific methods to study psychological questions is the investigation by Oskar Pfungst of the amazing abilities of Clever Hans, the wonder horse of Wilhelm von Osten. Pfungsts contribution comes from the period in which psychology was founded as an academic discipline, but it remains as significant now as it was when he reported his findings 100 years ago. In his comprehensive analysis of the performance of Clever Hans, Pfungst showed how the combination of naturalistic observation, hypothesis testing with field experimentation, and laboratory simulation can lead both to the discovery of new phenomena in psychology and to its explanation. The story of Clever Hans provides important lessons in scientific methodology. It also provides lessons in the importance of subtle cues as influences of what people (or other animals) do. This second point is still not widely appreciated, as is evident from the continuing success of mind readers. More distressingly, the failure to appreciate what Pfungst did and what he discovered leads people to accept clinical interventions, like facilitated communication for individuals with severe communication disorders, that do not withstand serious scrutiny. Thus, in relaying the story of Clever Hans and Pfungsts study of him, I hope to impart an appreciation of what psychologists should know as well as well as how they go about knowing it.
The story of Clever Hans begins with the belief of his owner, Herr Wilhelm von Osten, in the intelligence of horses. Von Osten was a mathematics teacher who, on his retirement, decided to convince others of his belief by teaching his horse, Hans, to communicate. He did so through an education program in which Hans learned to tap with his right front hoof as a way of counting out a numerical answer, to nod or shake his head to answer yes/no questions, and to move his head to give directions. For example, Hans would respond to the question What is 3 + 7? by tapping the ground 10 times. Herr von Osten and Clever Hans
Draft of January 18, 2004
Page 2
Hans ability to answer such questions came to the attention of Carl Stumpf, who was head of the Psychological Institute at the University of Berlin (von Osten lived in a suburb of Berlin). After several trips to see Hans, Stumpf headed up a commission (known as the September Commission, because that was the date of its report) to investigate whether the horses performance was due to some sort of trick or deception in which von Osten was deliberately providing cues or aids for Hans answers. The commission, consisting of two psychologists (including Stumpf) and people familiar either with instructional methods or with training animals, concluded that neither deception nor a trick of training was involved. The commission left open the question of how Hans was able to answer questions. Stumpf gave to one of his students, a young man named Oskar Pfungst, the task of investigating Hans ability in more depth, and, in a thorough and careful analysis, Pfungst finally figured out the basis for the cleverness of Hans.
Pfungsts Analysis: Step 1 - Experimentation Pfungst used the results of the September Commission as the starting point for his investigation. The commission members had observed Hans and von Osten, and they had seen no sign of cueing, either deliberate or unintentional. Thus, it appeared to them that either Hans was exhibiting intelligence, as von Osten claimed, or some undiscovered factor was aiding him. With its report, the commission provided an initial descriptive account of Hans performance under naturalistic conditions. To take the analysis of Hans ability further, Pfungst decided he needed more control of the overall situation, so he erected a large tent in the stableyard, creating, in effect, a large room in which to test Hans. The tent helped reduce the presence of distractions and allowed Pfungst to carry out repeated series of tests with few interruptions. To address the question of whether the answers were in Hans or in the questioner, Pfungst conducted tests in which the independent variable was whether or not the questioner knew the answer to the problem given to Hans. In making these his initial tests, Pfungst chose to Von Osten (l) and Pfungst (r) testing Hans.
Draft of January 18, 2004 test the following hypothesis: If Hans has the mental capacities attributed to him,
Page 3
then his test performance will not depend on his questioners knowing the answer. By choosing this hypothesis, Pfungst accomplished two notable results. First, he expressed the question of Hans intelligence in a manner that is clearly falsifiable. If Hans performance did depend on the questioners knowledge, then it was a logical certainty that Hans did not have the mental capacities attributed to him. Second, he created a way to compare two conditions that were physically identical and differed only in their psychological significance. Thus, the same person could ask Hans, at two different times, the same question, with the only difference being that the person knew the answer at one of these times and did not at the other. For example, the person might ask Hans What is the number on this card?, knowing what was on the card at the time of one test and not knowing at the time of another. This kind of comparison is particularly incisive and important for investigations, like those often found in psychology, in which the effective causal agent is either unknown or not well-specified. Because of its importance, it has a special designation, being known as a placebo control. The dependent variable for Pfungsts tests was the accuracy of Hans answers. What he found convinced him, as it has convinced almost everyone who reads his story, that Hans did not know the answers to the questions he was asked. From one series of test, for example, Pfungst reported that Hans was correct on 41 of 42 tests in which the questioner knew the answer, whereas he was correct on only 4 of 49 tests of the same sort but in which the questioner did not know the answer.1 This large difference in accuracy was also found in each of several different kinds of tests, showing that the result was not peculiar to a particular way of testing. Having decided that Hans depended on getting some kind of information from the questioner, Pfungst set about trying first to isolate the relevant sensory medium. Was Hans responding to visual cues, auditory cues, olfactory cues, tactile cues, or some combination of sensory dimensions? To identify the form of cue, Pfungst planned a series of tests in which he would systematically interfere with one or another sensory system to see if the interference
1
Note that Hans did get a few tests correct when the questioner did not know the answer, however. Pfungst treated these as lucky guesses.
Draft of January 18, 2004
Page 4
affected Hans. For example, to test for auditory cues, he planned to test Hans with and without ear plugs, on the idea that Hans performance would be impaired when he was wearing ear plugs if auditory information were important. For a variety of reasons, Pfungst began by examing the importance of visual cues, using tests with and without blinders on Hans. The form of his hypothesis was the following:
If Hans is relying on visual information, then his test performance will depend on how well he can see his questioner.
As it happened, the results of these tests clearly showed that wearing blinders interfered with Hans accuracy. For example, Pfungst reported that in a series of 102 tests with blinders, 35 tests were ones in which Hans could not see the questioner, 56 tests were ones in which he could see the questioner, and 11 were undecided as to whether Hans could see the questioner or not. When Hans could not see the questioner, he was correct on only of 2 the 35 tests, whereas when he could see the questioner, he was correct on 50 of 56 tests. Pfungst concluded that Hans was relying on visual cues. This conclusion is not, however, logically certain. Rather, in this test, the results support the hypothesis that visual information is important, but they do not prove it. The results were very useful, nonetheless, because they do rule out hypotheses about the possible role of other known sensory information, such as audition, olfaction, or tactile stimulation. That is, the corresponding hypothesis for auditory information is Hans with blinders on to limit visual cues.
If Hans is relying on auditory information, then his test performance will not depend on his wearing blinders.
Consequently, finding that Hans performance does depend on whether or not he is wearing
Draft of January 18, 2004
Page 5
blinders proves that he is not relying on auditory information. Thus, Pfungst could be certain that Hans was not using auditory, tactile or olfactory cues. One other aspect of Pfungsts tests of the role of visual information is worth noting, which is how difficult it was to be sure that Hans could not see the questioner. Just putting blinders on Hans was not enough, as Pfungst makes clear, to insure that Hans had no visual cues. For one thing, Hans tried hard to see the questioner, moving his head so the blinders did not block his view. For another, the horses available vision is difficult to estimate from casual inspection. Pfungst emphasized that he found effects of the blinders only for a set that were fairly large; with the much smaller pair he used initially, Hans was generally correct. Again, the lesson remains very relevant. James Randi, the magician, has emphasized that the use of blindfolds with human subjects is often an ineffective way to eliminate visual cues. Once, to prove a point to a reporter for New Yorker Magazine, for example, he drove around Manhattan while blindfolded. The reporter was duly impressed, although no information was provided on how much time was needed for the reporters arousal level to return to normal.
Pfungsts Analysis: Part 2 - Observation and Control Once he concluded that Hans relied on visual cues, Pfungst set about trying to determine what the cues were. He began by observing the interaction between horse and questioner very carefully. Unlike the members of the commission, who had come looking for evidence of fraud or deception, Pfungst was looking for some movement or motion of the questioner that reliably occurred just before Hans finished tapping. As he watched, over and over, what happened when von Osten posed a question to Hans and Hans tapped out his response, Pfungst finally discerned two kinds of subtle and almost imperceptible movements that seemed to cue Hans when to start and when to stop tapping. Pfungst noticed that von Osten would bend his head slightly, after he asked Hans a question, to look at Hans hoof and count the taps; Hans seemed to use this slight movement as a cue to start tapping. Then, when Hans reached the correct number of taps, von Osten would raise his head slightly, and Hans responded to this straightening up as a cue to stop tapping. It must be emphasized that these critical movements of von Osten were subtle and difficult to detect, partly because von Osten was a very animated and talkative person. Pfungst
Draft of January 18, 2004
Page 6
saw them only because he was paying careful attention at the moments they occurred. Having found what he thought was the critical cue for Hans to give his answer, Pfungst was careful to verify that others could also observe this cue (that it was publically verifiable) by arranging tests in which other people recorded the moment they saw a straightening up cue. These tests showed that others could also reliably identify the movement that caused Hans to stop tapping. Pfungst also showed that such movements could be observed in the two individuals, other than von Osten, who could reliably get answers from Hans (Pfungst himself was one of these individuals, and he commented on the fact that he was able only with difficulty to become aware of his own movements).2 However, the most compelling evidence Pfungst presented for his interpretation of Hans performance was that he could control, through changes in his posture, the number of times Hans tapped. In different tests, Pfungst was able to induce Hans to stop tapping before he reached the correct answer (something that almost never occurred naturally), to stop at the correct answer, or to continue tapping well past the correct answer. Pfungst could also control when Hans began to tap, a feature of the situation that had received much less attention, perhaps because many people as...