This preview shows page 1. Sign up to view the full content.
Unformatted text preview: M316 Chapter 9 Dr. Berg Producing Data: Experiments A study is an experiment when we actually do something to people, animals, or objects in order to observe the response. The distinction between explanatory and response variables is essential. Experiments First we need some terminology. Definition The individuals studied in an experiment are often called subjects, particularly when they are people. The explanatory variables in an experiment are often called factors. A treatment is any specific experimental condition applied to the subjects. If an experiment has several factors, a treatment is a combination of specific values of each factor. Example (9.1) Effects of Good Day Care Does day car help lowincome children stay in school and hold good jobs later in life? The Carolina Abecedarian Project has followed a group of children since 1972. The subjects are 111 people who in 1972 were healthy but lowincome black infants in Chapel Hill, North Carolina. All the infants received nutritional supplements and help from social workers. Half, chosen at random, were also placed in an intensive preschool program. The experiment compares these two treatments. There is a single factor, "preschool, yes or no." There are many response variables, recorded over more than 20 years, including academic test scores, college attendance, and employment. Example (9.2) Effects of TV Advertising What are the effects of repeated exposure to an advertising message? The answer may depend both on the length of the ad and on how often it is repeated. An experiment investigated this question using undergraduate students as subjects. All subjects saw a 40minute television program that included ads for a digital camera. Some subjects saw a 30second commercial; others, a 90second version. The same commercial was shown either 1, 3, or 5 times during the program. This experiment has two factors: length of the commercial, with 2 values, and repetitions, with 3 values. The 6 combinations of one value of each factor form 6 treatments. After viewing, all of the subjects answered questions about their recall of the ad, their attitude toward the camera, and their intention to purchase it. These are the response variables. Here is a diagram of the treatments. 1 M316 Chapter 9 Dr. Berg Exercise An Industrial Experiment An engineer investigates the effects of combinations of two temperatures (50 and 60 centigrade) and three stirring rates (60rpm, 90rpm, and 120rpm) on the yield of a chemical process. Two batches are processed for each combination and the yield is measured. a) How many factors are there? How many treatments are there? b) What are the explanatory and response variables? c) What are the individuals and how many are there? How to Experiment Badly Experiments are the preferred method of examining the effect of one variable on another. Proper design is as important for experiments as it is for surveys. Let's start with an example of a bad design. Example (9.3) An Uncontrolled Experiment A college regularly offers a review course to prepare candidates for the Graduate Management Admission Test (GMAT), which is required by most graduate business schools. This year, it offers only an online version of the course. The average GMAT score of students in the online course is 10% higher than the longtime average for those who took the classroom review course. Is the online course more effective? This experiment has a very simple design. A group of subjects (the students) were exposed to a treatment (the online course), and the outcome (GMAT scores) was observed. Here is the design: Subjects Online course GMAT score A closer look at the GMAT review course showed that the students in the online review were quite different from the students who in past years took the classroom course. In particular, they were older and more likely to be employed. We can't compare their performance with that of the undergraduates. The effect of online versus inclass instruction is confounded with the effect of lurking variables. 2 M316 Chapter 9 Dr. Berg Most laboratory experiments use a simple design like this: Subjects Treatment Measure response In the controlled environment of the laboratory, simple designs often work well. Field experiments and experiments with humans are exposed to more variable conditions and deal with more variable subjects. The results of such a simple design are often worthless. Randomized Comparative Experiments The remedy for confounding in Example 9.3 is to do a comparative experiment in which some students are taught in the classroom and other, similar students take the course online. The first group is called a control group. It is important that the groups being compared are similar. The solution is to use random assignment of subjects. Definition An experiment that uses both comparison of two or more treatments and chance assignment of subjects to treatments is a randomized comparative experiment. Example (9.4) Oncampus Versus Online The college decides to compare the progress of 25 oncampus students taught in the classroom with that of 25 students taught the same material online. Select the students who will be taught online by taking a simple random sample of size 25 from the 50 available subjects. The remaining 25 students form the control group. They will receive classroom instruction. The result is a randomized comparative experiment with two groups. Here is a diagram of the experimental design. The design is comparative because it compares two treatments. It is randomized because the subjects are assigned to the treatments by chance. Definition In a completely randomized experimental design, all the subjects are allocated at random among the treatments. 3 M316 Chapter 9 Dr. Berg Example (9.5) Conserving Energy Many utility companies have introduced programs to encourage energy conservation among their customers. An electric company considers placing electronic meters in households to show what the cost would be if the electricity use at that moment continued for a month. Will meters reduce electricity use? Would cheaper methods work almost as well? The company decides to conduct an experiment. One cheaper approach is to give customers a chart and information about monitoring their electricity use. The experiment compares these two approaches (meter, chart) and also a control. The control group of customers receives information about energy conservation but no help in monitoring electricity use. The response variable is total electricity used in a year. The company finds 60 singlefamily residences in the same city willing to participate, so it assigns 20 residences at random to each of the three treatments. Here is a diagram of the experimental design. Exercise (9.6) Can Tea Prevent Cataracts Eye cataracts are responsible for 40% of blindness around the world. Can drinking tea regularly slow the growth of cataracts? We can't experiment on people so we use rats as subjects. 18 young rats are injected with a cataract causing substance and randomly allocated among three treatments: drinking a black tea extract, a green tea extract, and a placebo. The growth of the cataracts is measured over the next six weeks. Diagram the experimental design. The Logic of Randomized Comparative Experiments Randomized comparative experiments are designed to give good evidence that differences in the treatments actually cause the differences we see in the response. The logic is as follows: 1 Random assignment of subjects forms groups that should be similar in all respects before the treatments are applied. 4 M316 Chapter 9 Dr. Berg 2 Comparative design ensures that influences other than the experimental treatments operate equally on all groups. 3 Therefore, differences in average response must be due either to the treatments or to the play of chance in the random assignment of subjects to treatments. Notice that we can never entirely eliminate doubt about differences in the groups assigned to the treatments. One would never trust an experiment with only one human subject in each treatment group. "Use enough subjects to reduce chance variation" is the third big idea of statistical design of experiments. Principles of Experimental Design 1 Control the effects of lurking variables on the response, most simply by comparing two or more treatments. 2 Randomizeuse impersonal chance to assign subjects to treatments. 3 Use enough subjects in each group to reduce chance variation in the results. We hope to see a difference in the responses so large that it is unlikely to happen just because of chance variation. We use the laws of probability to measure this. Definition An observed effect so large that it would rarely occur by chance is called statistically significant. The great advantage of randomized comparative experiments is that they can produce data that give good evidence for a causeandeffect relationship between the explanatory and response variables. We know that in general a strong association does not imply causation. A statistically significant association in data from a welldesigned experiment does imply causation. Exercise (9.8) Conserving Energy What would be the consequence of not including the control group? Cautions About Experimentation The logic of a randomized comparative experiment depends on our ability to treat all subjects identically in every way except for the actual treatments being compared Exercise 9.5 is a typical medical experiment. All of the subjects took the same tests and received the same medical attention. All of them took a pill every day, ginkgo in the treatment group and a placebo in the control group. A placebo is a dummy treatment. Many patients respond favorably to any treatment. The response 5 M316 Chapter 9 Dr. Berg to a dummy treatment is called the placebo effect. This was a blind experiment in that the subjects did not know who got the real treatment. In addition, it was double blind, because the doctors evaluating the response were also unaware of who got the real treatment. This eliminates any possible bias by the people measuring the response. Definition In a doubleblind experiment, neither the subjects nor the people who interact with them know which treatment each subject is receiving. The most serious potential weakness of experiments is lack of realism: the subjects or treatments or setting of an experiment may not realistically duplicate the conditions we really want to study. Example (9.6) Response to Advertising The study of television advertising in Example 9.2 showed a 40minute videotape to students who knew an experiment was going on. We can't be sure that the results apply to everyday television viewers. Example (9.7) Center Brake Lights Do those high center brake lights, required on all cars sold in the United states since 1986, really reduce rearend collisions? Randomized comparative experiments with fleets of rental and business cars, done before the lights were required, showed that the third brake light reduced rearend collisions by as much as 50%. Alas, requiring the third brake light in all cars led to only a 5% drop. What happened? When the experiments were done, third brake lights were a novelty and attracted the attention of other drivers. Matched Pairs and Other Block Designs Completely randomized designs are the simplest statistical designs for experiments. However, they are sometimes inferior to more elaborate designs. In particular, matching the subjects in various ways can produce more precise results than simple randomization. One common design that combines matching with randomization is the matched pairs design. A matched pairs design compares just two treatments. Choose pairs of subjects that are as closely matched as possible. Use chance to decide which subject in a pair gets the first treatment. The other subject in the pair gets the other treatment. Sometimes the "pair" consists of a single individual who gets both treatments one after the other. The order of treatments is randomized and the subject serves as his own control. 6 M316 Chapter 9 Dr. Berg Example Coke versus Pepsi The same subjects are asked to compare two colas. Some taste the cola in glass A first and then glass B, and some taste them in reverse order. Definition A block is a group of individuals that are known before the experiment to be similar in some way that is expected to affect the response to the treatments. In a block design, the random assignment of individuals to treatments is carried out separately within each block. The matched pairs design a simple form of block design. Here is another example. Example (9.9) Men, Women, and Advertising Women and men respond differently to advertising. An experiment to compare the effectiveness of three advertisements for the same product will want to look separately at the reactions of men and women. Instead of doing a complete randomization of all individuals to the three treatments, form a block of men and a block of women and then randomly assign the members of each block to one of the three treatments. Here is a diagram of the design. 7 ...
View Full Document
This note was uploaded on 09/14/2009 for the course CH 310 N taught by Professor Blocknack during the Fall '08 term at University of Texas at Austin.
- Fall '08