This preview shows page 1. Sign up to view the full content.
Unformatted text preview: Variables and T Tests/ANOVA Step 1: Start by assuming that probabilistic equivalence holds Null hypothesis = no difference Review: What is statistical significance?: Steps Step 2: Test the probability of that assumption being true Alpha or p value--the probability that the null hypothesis is true p = 1.0 = 100% chance that that there is no difference p = .05 = 5% chance that there is really no difference Steps Step 3: If p value is less than .05 Step 4: Accept the alternative hypothesis You reject the hypothesis that the effect is due to chance (the null hypothesis) That there is an effect That the groups really are different from zero That the correlation really is greater than zero Etc. Things that help Effect size "how big" is your result Sample size Correlation = close to 1 Groups = very much different from each other Called "power" = ability to correctly reject null More people = have a greater proportion of the population Put another way: If you tested every person in the population and found a r = .05 it is "statistically significant" by definition Clarification on Shading
Shade the portion which will allow you to reject the null hypothesis (5%--2SD) Note What "statistical significance" means What "statistical significance" doesn't That what you find in your sample is "really" in population mean Substantive meaning Can have a small effect be "significant" with large N Can have an effect from the "wrong" population Many different statistics are based on the same mathematics Different formulae because once it was Different formulae because once it was necessary to calculate results by hand Come from different intellectual traditions Difference mainly arises from different kinds of independent and dependent variables: Terminology Independent variable: What you are manipulating Effects: Also called a FACTOR Level: The values your independent level can take Null, Main, Interaction Terminology Dependent variable What you presume is being affected by your independent variable Also called effect or outcome variable Example... If you have an educational program you are testing for impact on academic achievement with various measures DV = Measures of academic achievement IV = Educational program What statistic do you use Question #1 how many independent variables do you have? Question #2 are your independent variables quantitative or categorical? Question #3 are you measuring the same subjects at different time points or not? If you can answer these questions, then you know which stats to use Ttests, ANOVA, and Review TTest Dependent variable is continuous ONE independent variable that is categorical Example: twogroup posttest only design Examine effect of ritalin and behavioral modification in ADHD Two groups: Exp = gets ritalin+behavioral mod, control=gets ritalin only Examine whether two groups are different on ADHD ADHD Score 80 75 70 65 60 55 50 45 40 35 30 Pretest Posttest r+behmod r only Where can we find differences? Group 1 and group 2 at pretest? Group 1 and group 2 at posttest? Change in group 1 from pretest to postest? Change in group 2 from pretest to postest?
80 75 70 65 60 55 50 45 40 35 30 Pretest Posttest r+behmod r only 1. 2. 3. 4. If the program is effective we want...
No group differences at pretest No change in control group from pre- post Group differences at posttest Change in program group from pre to post 80 75 70 65 60 55 50 45 40 35 30 Pretest Posttest r+behmod r only Make up some numbers
pretest MEAN 49 47 posttest MEAN 50 35 pretest STD DEV 6.9 7.0 posttest STD DEV 7.5 7.3 Control Program Between subjects = comparing different people Within subjects = comparing the same people Two kinds of ttests
1. Independent samples ttest: = BETWEEN SUBJECTS If different subjects are in different groups 1. Paired samples ttest: If the same subjects are measured twice = WITHIN SUBJECTS NOTE: ANY TIME YOU USE A PREPOST TEST DESIGN YOU HAVE A WITHIN SUBJECTS DESIGN BOTH kinds: Can only look at 2 levels in one independent variable at time (e.g. more than 2 need to break it up in separate analyses) 80 75 70 65 60 55 50 45 40 35 30 Independent Samples Comparison Program Pretest Posttest Independent Samples Different people in comparison and program groups 80 75 70 65 60 55 50 45 40 35 30 Paired t-tests: does group 1 change with program Comparison Program Pretest Posttest Paired: does group 2 change Same people over time Ttests Comparison t df p Independent samples t-tests 0.55 Pretest 3.24 Posttest Paired t-tests Control Program 49 49 49 49 .85 .01 .95 .01 0.25 3.33 T-tests
Comparison t df p No effects! Independent samples t-tests 0.55 Pretest 3.24 Posttest Paired t-tests Control Program 49 49 49 49 .85 .01 .95 .01 Low t-score High p-value 0.25 3.33 T-tests
Comparison t df p Significant effects Independent samples t-tests 0.55 Pretest 3.24 Posttest Paired t-tests Control Program 49 49 49 49 .85 .01 .95 .01
High t-score Low p-value 0.25 3.33 Looking at it another way... How to write it? Independent ttests in Table 3 suggest that the groups are not statistically different from zero at pretest t(49) = .55, p = .85 (two tailed) but are significantly different at post test t(49) = 3.24, p = .01 (two tailed). Moreover, paired ttests, also presented in Table 3 suggest that ADHD symptoms significantly declined in the program group t(49) = 3.33, p = .01 (two tailed), but not in the control group t(49) = .25, p = .95 (two tailed). Summary Between subjects yes Within subjects no #indep variables 1 #levels test 2 Indep. T-test no yes 1 2 Paired T-test A wrinkle... REMEMBER: Can only look at 2 levels at a time So if you have 3 time points, can't do all of them at once (have to look at 2 of the 3 in any one ttest) So if you have 3 groups, can only look at 2 groups in any one ttest Scenario #1 Add more measurement occasions (more posttests)
80 75 70 65 60 55 50 45 40 35 30 Pretest Posttest1 Posttest2 Comparison Program Scenario #1 What kind of design is this? What would you have to do to Table 3 How many paired t-tests? How many independent samples t-tests?
80 75 70 65 60 55 50 45 40 35 30 Pretest Posttest1 Posttest2 Comparison Program Scenario #1 What would you have to do to Table 3 How many paired ttests = 6 How many independent samples ttests? = 3 Group 1 vs. group 2 at Pretest, postest1, posttest2 Group 1: Pretest posttest1, Posttest1 posttest 2 Pretest posttest 2 Group 1: Pretest posttest1, Posttest1 posttest 2 Pretest posttest 2 80 75 70 65 60 55 50 45 40 35 30 Pretest Posttest1 Posttest2 Comparison Program Scenario #2 Add more groups 80 75 70 65 60 55 50 45 40 35 30 Pretest Posttest Comparison Program #1 Program#2 Scenario #2 What kind of design is this? What would you have to do to Table 3 How many paired t-tests? How many independent samples t-tests?
80 75 70 65 60 55 50 45 40 35 30 Pretest Posttest Comparison Program #1 Program#2 Scenario #2 What kind of design is this? What would you have to do to Table 3 How many paired t-tests = 3 Group 1: pre- post Group 2: pre-post Group 3: pre-post How many independent samples t-tests? = 6 Pretest: group1 vs. group 2, group 2 vs. group 3, group1 vs. group 3 Posttest: group1 vs. group 2, group 2 vs. group 3, group1 vs. group 3 Now: Think about what happens if you have Why we've been asking for simpler More than 1 DV 3 or more groups, 3 or more measurement occasions Same logic applies, just more complicated designs! Why ANOVA exists! Brief Review and Clarifications Correlation Pvalue vs. rvalue Pvalue is related to confidence in ability to reject the null hypothesis Low pvalue (p < .05) Reject the null hypothesis; Accept the alternative Less than a 5% chance that the result is a chance occurrence High pvalue (p >.05) Fail to reject the null hypothesis; Reject the alternative Pvalue vs. rvalue rvalue is related to the relationship between two continuous variables Positive rvalue Positive relationship If the value for one is high/low, the value for the other is also high/low Negative rvalue Negative relationship If the value of one is high/low, the value for the other is the opposite or the inverse. Statistics Cheat Sheet ...
View Full Document
This note was uploaded on 08/26/2009 for the course BB H 310W taught by Professor Saltsman,brian during the Spring '07 term at Pennsylvania State University, University Park.
- Spring '07