9 Chapter The Significance of the Difference between Means The t-test for comparing the difference between two means Tests the hypothesis that the mean scores on some interval- or ratio-scaled variables will be significantly different for the two independent samples or groups How far apart do the two mean s have to be so that one can say the difference is not due to chance/sampling error Answer: resort to sampling distribution Null Hypothesis: difference between sample means Assume there is no difference in the means of the populations from which the 2 samples were drawn: i.e. assume the means of the 2 populations are the same Ho : 1 - 2 = 0 Slight difference between means will occur due to sampling error t= Mean 1 - Mean 2______ Variability of random means t = X1 - X2 = S X1-X2 *If one surveys many different samples, there will be a whole array of differences between means: X1 - X2, X3 - X4, X5 - X6 and so forth 1. The differences between means are arrayed as a sampling distribution 2. The mean of the sampling distribution would be 0 3. The batch of differences would be normally distributed around the mean of the differences The distribution of differences in a normal distribution, therefore: 1. 68.23% of the differences would fall between -1 and +1 standard deviations from the mean 2. 95% of all the differences would fall between -1.96 and +1.96 standard deviations from the mean 3. 99% of the differences would fall between -2.58 and +2.58 standard deviations from the mean Probability statements concerning the normal curve would apply 1. p = .68 that any difference would fall between u diff + or - standard deviation 2. p = .95 that any difference would fall between u diff + or - 1.96 standard deviations 3. p = .05 that any difference would fall outside the same interval, u diff = or - 1.96 standard deviations The question we ask then is where in the sampling distribution of differences does our obtained difference (between the two means) lie? If it is near the mean, i/e. between -1.96 and +1.96, we say the difference is due to sample error. If the difference is large and falls outside the area of -1.96 and +1.96, we think the difference may not be do to sample error, i.e. it is a rare occurrence The procedure: calculate the z-score for the difference between the two means and find its location in the sampling distribution of differences *USE TABLE A. FIND THE X-SCORE, THEN LOOK TO THE RIGHT TO FIND AREA UNDER THE NORMAL CURVE. Subtract this area from .50 (half of the distribution you would expect to be under the normal curve to the right or left) and you will get the probability of getting your result by chance. *If inside the specified range of +/- 1.96, inside the .05 level, accept null hypothesis difference between means___________ combined standard error of difference between means
*If outside the specified range of +/- 1.96, outside the .05 level, reject null hypothesis Summary of steps for determining the significance of a difference between 2 means: 1. Assume a null hypothesis; i.e., that the means of the two populations from the samples drawn are equal 2. Calculate the z-score for your obtained difference between the means of the 2 samples and locate it in the sampling distribution of differences whose mean u diff is 0 3. If the resulting z-score falls between u diff = +/- 1.96 assume the difference is due to sampling error 4. If the resulting score is outside that range, reject the null hypothesis *Note that we are looking at the difference between 2 means, not the difference of a score from the mean. In a sense the means have now become like scores and we are comparing them. The formula for the z-score necessitates use of the standard error of the difference between means. This is the standard deviation of the hypothetical sampling distribution of differences. The standard error of the mean of a distribution is calculated from the population standard deviation. Since this is rarely known, we use a formula using the standard deviations of the 2 samples. When standard deviations of the population are unknown, we cannot use the normal distribution and z-score values of 1.96 and 2.58. Instead us t-distribution and calculate t-values t-value formula is the same as z-score except for the denominator, which substitutes the standard deviations of the 2 samples for the population standard deviation t-distribution looks like normal curve except tails are higher-we must go further out, past +/- 1.96, say +/- 2.78, to find values to mark off the 5% and 1% regions *As N gets larger, the distribution of t approaches that of the normal curve. With combined sample size of 30 or more, the distributions are approximately identical. Degrees of freedom: in a sample is the number of observations that are free to vary Df = N-1 A restriction on a set of numbers means that all but one can vary Ex: Sum of X = 12 and there are 5 numbers, 4 numbers can vary, but the 5th must be such that the sum of all 5 numbers = 12so, 3+2+4+7 = 16, thus the 5th number be must -4 to get the sum 12. Since the t-test has 2 samples, add the degrees of freedom for the 2 samples (N1 - 1) + (N2 - 10 or N1 + N2 - 2 USE TABLE D TO FIND WHAT t-VALUES YOU NEED TO GET IN ORDER FOR YOUR DATA TO BE SIGNIFICANT AT THE .05 LEVEL, THE .01 LEVEL AND THE .001 LEVEL Significance Levels 1. If a difference would occur 5% of the time or less by sampling error (but is not large enough to each the .01 level) we state The Null Hypothesis and Other Hypotheses Rejection of the Null Hypothesis does not necessarily make an experimental hypothesis true The researcher cannot prove why a difference exists One does not always assume a null hypothesis equal to zero Could be based on prior data the size of difference between 2 measuresBook example: difference in #s of bushels of corn produced by 2 seed varieties in an earlier study versus the differences in #s obtained in a later study Assumptions underlying the t test
Scores must be interval or ratio Scores must be measures on random samples from the respective populations Populations from which the samples were drawn must be normally distributedpsychological and physiological characteristics are normally distributed in population (not saying samples are normally distributed) 4. Populations from which the samples were drawn must have approximately the same variability (homogeneity of variance). (Can "eyeball" the standard deviations and if one seems twice as large as other may question the homogeneity of variance). (Can also use SSPS computer solution) The t-test is robust i.e., will give fairly accurate results even if assumptions are violated to a certain degree If in doubt, use more stringent significance level: .01 instead of .05 TESTING FOR SIGNIFICANT DIFFERENCE; THE t-TEST FOR TWO CORRELATED SAMPLES Matched samples: don't depend on random assignments to get samples. Instead match, make sure are equal on variables to be measured. 1. matched pairs: match on some variable that correlates highly with what we are measuring Example: may want to match on college entrance exam scores 2. split litters: animal experiments take group example members of litter of kittens, put one member in the experimental group and other members in the control group 3. co-twin controls: human twins, one to experimental groups one to control group Repeated measures of the same subject: same person serves in both groups, so both groups really in sense are same persons Data from above are "correlated" When calculating t-tests with correlated data, we work with the differences between pairs of observations N = the number of pairs D = sum of the differences (D) column Errors in Making Decisions Type I Error: when there is no difference between the means of the populations from which the samples were drawn, and we made a mistake by calling our obtained difference a significant one instead of attributing it to sampling error. Avoid: make significance level more stringent, .01 or .001 instead of .05 Type II Error: the null hypothesis is accepted when actually it is false, there is a real difference. P= As we decrease the probability of a type I error we increase the probability of a Type II error. Power of a test: the ability of a test to reject the null hypothesis when it is false The more power a statistical test has, the more likely it is to detect significant differences if they exist. Use common sense in selecting the significance level or alpha, which results in a greater propensity of one type error or another If you must make a decision between 2 alternatives, choose a less stringent level If you want to make sure you don't get a weird/bizarre result by chance, choose more stringent level
1. 2. 3.
Correlated versus Uncorrelated t test and Type II Errors The larger t obtained with the correlated t test means one is more likely to reject the null hypothesis and less likely to make a Type II error-you are more likely to pick up a difference The Df for correlated samples is 1 less than for uncorrelated samples Uncorrelated Df = (N1 - 1) + (N2 - 1) Correlated Df = (N - 1) (where N is the number of pairs)
One-tailed versus Two-Tailed Tests Two-tailed test: null hypothesis will be rejected if the t-value is wither to the extreme left or the extreme right of the sampling distribution We don't state the direction of the difference when we post the null hypothesis, only if here is a difference One-tailed test: If one predicts the direction of the difference, there is a "directional" hypothesis Region of rejection is at one end of the sampling distribution only Testing Hypotheses about a Single Sample Mean When you test whether there is a significant difference between the mean of a sample and some hypothesized population mean, but don't know the population standard deviation, you may use the sample standard deviation, the standard error of the mean instead of the population standard deviation. *Must use the t-distribution and t values instead of the normal curve with z values