This preview shows page 1. Sign up to view the full content.
Unformatted text preview: M316 Chapter 8 Dr. Berg Producing Data: Sampling We have looked at what to do with data: exploratory data analysis. Now we examine methods for collecting that data. Observation versus Experiment Sometimes we merely gather information about a population, and sometimes we impose a treatment in order to observe the response. Definition Observation versus Experiment An observational study observes individuals and measures variable of interest but does not attempt to influence the responses. The purpose of an observational study is to describe some group or situation. An experiment, on the other hand, deliberately imposes some treatment on individuals in order to observe their responses. The purpose of an experiment is to study whether the treatment causes a change in the response. Observational studies are essential sources of data about topics from the opinions of voters to the behavior of animals in the wild, but they are a poor way to gauge the effect of an intervention. To see the response to a change, we must actually impose the change. Example (8.1) Hormone Replacement Therapy Should women take hormones such as estrogen after menopause, when natural production of these hormones ends? In 1992, several major medical organizations said "Yes." In particular, women who took hormones seemed to reduce their risk of a heart attack by 35% to 50%. The risks of taking hormones appeared small compared with the benefits. The evidence in favor of hormone replacement came from a number of observational studies that compared women who were taking hormones with others who were not. But women who choose to take hormones are very different from women who do not: they are richer and better educated and see doctors more often. Experiments don't let women decide what to do. They assign women to either hormone replacement or to a dummy pill. Women from different age groups were randomly assigned for treatment or placebo. By 2002, several experiments agreed that hormone replacement does not reduce heart attacks. Hormone replacement therapy is no longer recommended. The effects of the therapy were mixed up with the characteristics of women who chose the therapy. 1 M316 Chapter 8 Dr. Berg Definition Two variables (explanatory or lurking) are confounded when their effects on a response variable cannot be distinguished from each other. Observational studies of the effect of one variable on another often fail because the explanatory variable is confounded with lurking variables. A well designed experiment takes steps to defeat confounding. Example (8.2) Wine, Beer, or Spirits? Moderate use of alcohol is associated with better health. Observational studies suggest that drinking wine rather than beer or spirits confers added benefits. But people who prefer wine are different from those who drink mainly beer or spirits. Wine drinkers as a group are richer and better educated. They eat more fruits and vegetables and less fried food, smoke less, and so on. A large study therefore concludes: "The apparent health benefits of wine compared with other alcoholic beverages, as described by others, may be a result of confounding by dietary habits and other lifestyle factors." Exercise (8.1) Cell Phones and Brain Cancer Investigators compared 469 brain cancer victims with 469 healthy people who matched them in other characteristics. They were asked about their cell phone use. It was found that there is no association between cell phone use and brain cancer. Is this an observational study or an experiment? What are the explanatory and response variables? Exercise (8.2) Teaching Economics An educational software company wants to compare the effectiveness of its computer animation for teaching about supply and demand curves with that of a textbook presentation. The company tests the economic knowledge of a number of firstyear college students, then divides them into two groups. One group uses the animation, and the other studies the text. The company retests all the students and compares the increase in economic understanding in the two groups. Is this an observational study or an experiment? What are the explanatory and response variables? Exercise (8.3) TV Viewing and Aggression A typical hour of primetime television shows three to five violent acts. Research show that there is a clear association between time spent watching TV and aggressive behavior by adolescents. Nonetheless, it is hard to conclude that watching TV causes aggression. What are possible lurking variables describing an adolescent's home life that may be confounded with how much TV is watched? 2 M316 Sampling Chapter 8 Dr. Berg Sampling is used to gather information of interest about a population when it is not practical to collect the data for every individual. Definition The population in a statistical study is the entire group of individuals about which we want information. A sample is a part of the population from which we actually collect information. We use a sample to draw conclusions about the entire population. A sampling design describes exactly how to choose a sample from the population. Choosing a representative sample from a large and varied population is not easy. The first step in a sample survey is to say exactly what population we want to describe. The second step is to say exactly what we want to measure, that is, give exact definitions of the variables. The following example illustrates that these preliminary steps can be complicated. Example (8.3) The Current Population Survey The most important government survey in the United States is the monthly Current Population Survey (CPS). The CPS contacts about 60,000 households each month. It measures the monthly unemployment rate among other things. To measure unemployment, we must first specify the population we want to describe. Which age groups will we include? Will we include illegal aliens or people in prisons? The CPS defines its population as all U.S. residents (whether citizens or not) 16 years of age and over who are civilians and are not in an institution such as prison. The second question is harder: what does it mean to be "unemployed"? A fulltime student for example should not be called unemployed. You are considered to be in the labor force if you are available to work and worked or looked for work in the past four weeks. If you are in the labor force, you are employed if you did any work for pay or in a business you own during the previous week. Exercise (8.4) Sampling Students A political scientist wants to know how college students feel about the Social Security System. She obtains a list of the 3456 undergraduates at her college and mails a questionnaire to 250 students selected randomly. Only 104 questionnaires are returned. What is the population? What is the sample? Exercise (8.6) Customer Satisfaction A department store mails a customer satisfaction survey to people who make credit card purchases at the store. This month, 45,000 people made credit card 3 M316 Chapter 8 Dr. Berg purchases. Surveys were mailed to 1000 of these who were randomly chosen and 137 returned the form. What is the population? What is the sample? How to Sample Badly How do we choose a sample that is truly representative of the population we wish to study? There are numerous pitfalls. Exercise Callin Opinion Polls The ABC program "Nightline" once asked viewers to call in and answer the question: "Should the UN continue to be headquartered in the US?" Of more than 186,000 callers, 67% said no. A different properly designed survey indicated that 72% wanted the UN to stay. Why was there such a difference? Example (8.4) Sampling at the Mall A sample of mall shoppers is fast and cheap. But people who shop at malls tend to be more prosperous than typical Americans. They are also more likely to be teenagers or retired. Interviews at shopping malls will almost surely overrepresent middleclass and retired people and underrepresent the poor. This is a systematic error caused by bad sampling design. Definition A sample selected by taking the members of the population that are easiest to reach is called a convenience sample. A design of a statistical study is biased if it systematically favors certain outcomes. A voluntary response sample consists of people who choose themselves by responding to a broad appeal. Voluntary response samples are biased because people with strong opinions are most likely to respond. Exercise (8.7) Sampling on Campus You see a female student standing in front of the student center, now and then stopping other students to ask them questions. She says that she is collecting student opinions for a class assignment. Explain why this sampling method is almost certainly biased. Exercise (8.8) More Sampling on Campus Your college wants to gather student opinion s about parking for students on campus. It isn't practical to contact all students. Give an example of a poor sample design that (a) uses voluntary response, and (b) does not use voluntary response. 4 M316 Simple Random Samples Chapter 8 Dr. Berg In a voluntary response sample, people choose themselves. In a convenience sample, the interviewer makes the choice. In both cases, personal choice can produce bias. The remedy is to allow impersonal chance to choose the sample. The simplest way to do this is to "draw names from a hat". An SRS of sufficient size removes the problem of bias. Definition A simple random sample (SRS) of size n consists of n individuals from the population chosen in such a way that every set of n individuals has an equal chance to be the sample actually selected. An SRS not only gives each individual an equal chance to be chosen but also gives every possible sample of that size an equal chance to be chosen. There are other random sampling designs that give each individual, but not each sample, an equal chance. Example (8.5) The Future of the Environment "Do you think the condition of the environment for the next generation will be better, worse, or about the same as it is now?" When the New York Times and CBS News asked this question of 1052 adults, 57% said "worse" and just 11% said "better." Can we trust the opinions of this sample to fairly represent the opinions of all adults? Here's part of the statement by the Times on "How the Poll Was Conducted": The latest New York Times/CBS News poll is based on telephone interviews conducted April 20 through April 24 with 1,052 adults throughout the United States. The sample of telephone exchanges called was randomly selected by a computer from a complete list of more than 42,000 active residential exchanges across the country. The exchanges were chosen so as to ensure that each region of the country was represented in proportion to its population. Within each exchange, random digits were added to form a complete telephone number, thus permitting access to listed and unlisted numbers alike. Within each household, one adult was designated by a random procedure to be the respondent for the survey. This is a good description of the most common method for choosing national samples, called random digit dialing. This method is not without its problems, and we will discuss these later. Now we look at the most general method of producing an SRS. Software can be used to select an SRS, or a table of random digits can be used. 5 M316 Chapter 8 Dr. Berg Definition A table of random digits is a long string of the digits 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 with these two properties: 1 Each entry in the table is equally likely to be any of the 10 digits 0 through 9. 2 The entries are independent of each other. That is, knowledge of one part of the table gives no information about any other part. Table B in the back of the book is a table of random digits with the digits grouped in fives to make it easier to read. There are two steps in using the table to choose a simple random sample. Procedure for Using Table B Label: Give each member of the population a numerical label of the same length. Table: To choose an SRS, read from Table B successive groups of digits of the same length as the labels. Your sample contains the individuals whose labels you find in the table. Begin the labels with 1, 01, or 001 for labels of length 1, 2, and 3 respectively. When the table gives you digits that do not correspond to the labeling of one of the individuals, move to the next group of digits. The table can be used going across the rows or down the columns, but going across the rows is the norm. The line where you begin can be randomly chosen, and different lines should be used for different samples. Homework problems often tell you which line to use first. Exercise (8.6) Apartment Living You are planning a report on apartment living in a college town. You decide to select three apartment complexes at random for indepth interviews with residents. Use Table B to select a simple random sample of three of the following complexes. Start at line117. Ashley Oaks Country View Mayfair Village Bay Pointe Country Villa Nobb Hill Beau Jardin Crestview Pemberly Courts Bluffs DelLynn Peppermill Brandon Place Fairington Pheasant Run Briarwood Fairway Knolls River Walk Brownstone Fowler Sagamore Ridge Burberry Place Franklin Park Salem Courthouse Cambridge Georgetown Village Square Chauncey Village Greenacres Waterford Court Country Squire Lake House Williamsburg 6 M316 Other Sampling Designs Chapter 8 Dr. Berg Designs for random sampling from large populations spread over a large area are usually more complex than an SRS. For example, it is common to sample important groups within the population separately, then combine these samples. Definition To select a stratified random sample, firs classify the population into groups of similar individuals, called strata. Then choose a separate SRS in each stratum and combine these SRSs to form the full sample. Choose the strata based on facts known before the sample is taken. For example, a population of election districts might be divided into urban, suburban, and rural strata. Example (8.6) Seat Belt Use in Hawaii Each state conducts an annual survey of seat belt use by drivers, following guidelines set up by the federal government. The guidelines require random sampling. Seat belt use is observed at randomly chosen road locations at random times during daylight hours. The locations are not an SRS of all locations in the state but rather a stratified sample using the state's counties as strata. In Hawaii, the counties are the islands that make up the state's territory. The seat belt survey sample consists of 135 road locations in the four most populated islands: 66 in Oahu, 24 in Maui, 23 in Hawaii, and 22 in Kauai. The sample sizes on the islands are proportional to the amount of road traffic. Cautions About Sample Surveys When the population consists of humans, accurate information from a sample requires more than a good sample design. To begin, we need an accurate and complete list of the population. Because such a list is rarely available, most samples suffer from some degree of undercoverage. A sample survey of households, for example, will miss homeless people, prison inmates and students living in dormitories. A more serious source of bias is nonresponse, when a selected individual cannot be contacted or refuses to cooperate. Nonresponse is higher in urban areas, so pollsters often compensate by selecting more individuals in urban areas. In addition, the behavior of the respondents or the interviewer can cause response bias in sample results. People often lie about voting, sex, cheating, etc. The race or gender of the interviewer can influence responses to questions about race relations or attitudes toward feminism. 7 M316 Chapter 8 Dr. Berg The wording of questions is the most important influence on the answers given to a sample survey. Confusing or leading questions can introduce strong bias. Even the order of questions can have an influence. Example (8.9) Are You Happy? Ask a sample of college students these two questions: 1 "How happy are you with your life in general?" 2 "How many dates did you have last month?" When asked in this order, the correlation is r = -0.012 , but in the reverse order the correlation is r = 0.66 . Asking a question that brings dating to mind makes dating success a big factor in happiness. Inference About a Population Since we use samples to make inferences about the larger population, the bias introduced by these problems with sample design and other things can destroy the validity of the conclusions. To make valid inferences, we need carefully designed studies and experiments. 8 ...
View Full Document
- Fall '08