Statistics
Descriptive Inferential Statistics: Are used to describe, summarize and simplify data. Provides a single (typically) numeric value to summarize some aspect of the overall data set.
Inferential Statistics: Are used to infer the status of a question (about descriptive statistics) in a full population of individuals based on a sample from that population. Answers from inferential statistical are probabilistic. In other words, all answers have the potential to be wrong and you will provide an index of that probability along with your 1 results.
Populations vs. Samples
Population: Any clearly defined set of objects or events (people, occurrences, animals, etc.). Populations usually represent all events in a particular class (e.g., all college students, all alcoholics, all depressed people). Researchers usually want to make statements about populations. Sample: Any subset drawn from a population. Researchers work with samples of subjects and draw inferences about the larger population. Sampling error: Chance variation in statistics from different samples drawn from the same population.
2
Raw MAS scores for "population" of my students
Descriptive Statistics N Statistic TOTAL Valid N (listwise) Minimum Statistic Maximum Statistic Mean Statistic Std. Deviation Statistic Skewness Statistic Std. Error Kurtosis Statistic Std. Error
73 73
2
40
18.66
9.569
.253
.281
-.788
.555
10
8
Frequency
6
4
2
0 0-3 3-6 6-9 12 - 15 9 - 12 18 - 21 24 - 27 30 - 33 36 - 39 39 - 42 15 - 18 21 - 24 27 - 30 33 - 36
MAS Total
3
Sampling Distribution of the Mean
You can construct a sampling distribution for every sample statistic (e.g., mean, SD, min, max) For the mean, you can think of the sampling distribution conceptually as follows: 1. Imagine drawing many samples (lets say 20 but in theory, the sampling distribution is infinite) of 5 participants from your population 1. Next, calculate the mean for each of these samples 1. Finally, create a histogram (or frequency polygon) of these sample means
NOTE: The distinction between raw score and sample distributions is very important to keep clear in your mind! 4
Samples of MAS Anxiety Scores
Descriptive Statistics N Statistic 5 5 Minimum Statistic 11.00 Maximum Statistic 20.00 Mean Statistic 14.4000 Std. Deviation Statistic 3.64692 Skewness Statistic Std. Error 1.064 .913 Kurtosis Statistic Std. Error .202 2.000
TOTAL Valid N (listwise)
Descriptive Statistics N Statistic 5 5 Minimum Statistic 13.00 Maximum Statistic 36.00 Mean Statistic 20.2000 Std. Deviation Statistic 9.28440 Skewness Statistic Std. Error 1.730 .913 Kurtosis Statistic Std. Error 3.189 2.000
TOTAL Valid N (listwise)
Descriptive Statistics N Statistic 5 5 Minimum Statistic 10.00 Maximum Statistic 36.00 Mean Statistic 20.0000 Std. Deviation Statistic 10.27132 Skewness Statistic Std. Error .994 .913 Kurtosis Statistic Std. Error .943 2.000
TOTAL Valid N (listwise)
Descriptive Statistics N Statistic 5 5 Minimum Statistic 14.00 Maximum Statistic 30.00 Mean Statistic 23.2000 Std. Deviation Statistic 6.97854 Skewness Statistic Std. Error -.495 .913 Kurtosis Statistic Std. Error -2.110 2.000
TOTAL Valid N (listwise)
5
Sampling Distribution of the Mean
Descriptive Statistics N SAMPMEAN Valid N (listwise) Minimum Maximum Mean Std. Deviation
20 20
8
13.13
26.10 18.6089
3.74693
6
Frequency
4
2
0 0-3 3-6 6-9 12 - 15 9 - 12 18 - 21 24 - 27 30 - 33 36 - 39 39 - 42 15 - 18 21 - 24 27 - 30 33 - 36
MAS Sample Means
6
Sampling Distribution of the Mean
What what will the mean of the sample means be?
7
Sampling Distribution of the Mean
The mean is an unbiased estimator: The mean of the sample means will approximate the mean of the population. It will neither systematically under- or overestimate the population mean.
Descriptive Statistics N Statistic TOTAL Valid N (listwise) Minimum Statistic Maximum Statistic Mean Statistic Std. Deviation Statistic Skewness Statistic Std. Error Kurtosis Statistic Std. Error
73 73
2
40
18.66
9.569
.253
.281
-.788
.555
Descriptive Statistics N SAMPMEAN Valid N (listwise) Minimum Maximum Mean Std. Deviation
20 20
13.13
26.10 18.6089
3.74693
The SD is also an unbiased estimator (when you use the correct n-1 denominator). In other words, the mean of the sample SDs will approximate the population SD.
8
Sampling Error (SE)
Descriptive Statistics N SAMPMEAN Valid N (listwise)
8
Minimum
Maximum
Mean
Std. Deviation
20 20
13.13
26.10 18.6089
3.74693
6
Frequency
4
2
0 0-3 3-6 6-9 12 - 15 9 - 12 18 - 21 24 - 27 30 - 33 36 - 39 39 - 42 15 - 18 21 - 24 27 - 30 33 - 36
MAS Sample Means
9
Sampling Error (SE)
The standard deviation of the distribution of sample means (which is referred to as sampling error) is equal to: SDP/sqrt(NS) Sampling error is simply the variation in the sample mean. It is called error b/c you can think of it as the degree to which you may be "off" if you use an individual sample to estimate a population mean. Therefore, b/c we use sample (descriptive) statistics to estimate population parameters, we would like to minimize sampling error.
10
Sampling Error
What factors affect the size of the sampling error (hint: consider the formula on the preceding slide)?
11
Factors that Affect Sampling Error (SE)
Variation on the variable in the population is caused by two things. What might they be?
What is the relationship between population variability (SDP) and SE?
What would happened to SE if there was no variation in population scores?
12
Factors that Affect Sampling Error (SE)
What is the relationship between sample size and SE?
What would the SE be if the sample size = population size?
What would happen if the samples contained only 1 participant?
13
Shape of the Sampling Distribution
Central Limit Theorem: The shape of the sampling distribution approaches normal as N increases. Roughly normal even for moderate sample sizes assuming that the original distribution isn't really weird (i.e., nonnormal).
14
Brief Review....
Distribution of raw MAS scores in the population (N=73)
10
Distribution of sample means (sampling distribution of 20 samples of 5 students)
8
8 6
Frequency
6
Frequency
4
4
2 2
0 0-3 3-6 6-9 12 - 15 9 - 12 18 - 21 24 - 27 30 - 33 36 - 39 39 - 42 15 - 18 21 - 24 27 - 30 33 - 36
0 0-3 3-6 6-9 12 - 15 9 - 12 18 - 21 24 - 27 30 - 33 36 - 39 39 - 42 15 - 18 21 - 24 27 - 30 33 - 36
MAS Total
MAS Sample Means
Descriptive Statistics N TOTAL Valid N (listwise) Minimum Maximum Mean Std. Deviation
Descriptive Statistics N SAMPMEAN Valid N (listwise) Minimum Maximum Mean Std. Deviation
73 73
2
40
18.66
9.569
20 20
13.13
26.10 18.6089
3.74693
15
Brief Review....
What is the mean of the sampling distribution of the mean
What is the SD of the sampling distribution of the mean What factors affect the sampling error?
What is the shape of sampling distribution of the mean?
16
Important Concepts About Normal Distributions
1. There is a whole family of distributions that are "normal" in shape. All of these distributions can be described by the same mathematical formula (using different means and standard deviations).
17
Family of Normal distributions I 1 , , , Probability , I , , , , &
X score
Important Concepts About Normal Distributions
1. There is a whole family of distributions that are "normal" in shape. All of these distributions can be described by the same mathematical formula (using different means and standard deviations). 1. Many variables in nature (height, intelligence, speed, etc.) are "approximately" normally distributed. The sampling distribution of the mean of these variables is normally distributed (Central limit theorem)
19
10
8
Frequency
6
4
2
0 0-3 3-6 6-9 12 - 15 9 - 12 18 - 21 24 - 27 30 - 33 36 - 39 39 - 42 15 - 18 21 - 24 27 - 30 33 - 36
MAS Total
Important Concepts About Normal Distributions
1. There is a whole family of distributions that are "normal" in shape. All of these distributions can be described by the same mathematical formula (using different means and standard deviations). 1. Many variables in nature (height, intelligence, speed, etc.) are "approximately normally distributed. The sampling distribution of the mean of these variables is normally distributed (Central limit theorem) 1. Specific information about the "area under the normal curve" is known, and allows us to calculate how probable any score in a distribution is.
21
Probability of Scores
What is the probability of getting a score of 39 or greater?
10
8
Frequency
6
4
2
0 0-3 3-6 6-9 12 - 15 9 - 12 18 - 21 24 - 27 30 - 33 36 - 39 39 - 42 15 - 18 21 - 24 27 - 30 33 - 36
MAS Total
22
Important Concepts About Normal Distributions
1. There is a whole family of distributions that are "normal" in shape. All of these distributions can be described by the same mathematical formula (using different means and standard deviations). 1. Many variables in nature (height, intelligence, speed, etc.) are "approximately normally distributed. The sampling distribution of the mean of these variables is normally distributed (Central limit theorem) 1. Specific information about the "area under the normal curve" is known, and allows us to calculate how probable any score in a distribution is. 1. We can use the normal distribution to determine how probable any score on these variables is. 1. One particular normal distribution (The Standard Normal or zdistribution) is particularly helpful. The z-distribution has a mean of 0 and a standard deviation of 1. The exact probabilities associated with all z-scores has been calculated and tabled. 23
68%
95%
Standard Normal Distribution
99%
x 3 x Y Proportion x ~ x x H z s ' m _ f ` l F
z-score
24
Probability using Standard Normal Distribution
Can use the standard normal distribution to determine how probable any particular score will be. This score can be a raw score from the population distribution of scores (if population distribution is normal or some other known distribution) This "score" can also be a sample mean (i.e., a descriptive statistic) from the sampling distribution of the mean (and we know that sampling distribution will be normal!)
25
Probability using Standard Normal Distribution
What is the probability of getting a score of 39 or greater?
10
8
Frequency
6
4
2
0 0-3 3-6 6-9 12 - 15 9 - 12 18 - 21 24 - 27 30 - 33 36 - 39 39 - 42 15 - 18 21 - 24 27 - 30 33 - 36
MAS Total
26
Z 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2.0 2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8 2.9 3.0
0.00 0.5000 0.5398 0.5793 0.6179 0.6554 0.6915 0.7257 0.7580 0.7881 0.8159 0.8413 0.8643 0.8849 0.9032 0.9192 0.9332 0.9452 0.9554 0.9641 0.9713 0.9772 0.9821 0.9861 0.9893 0.9918 0.9938 0.9953 0.9965 0.9974 0.9981 0.9987
0.01 0.5040 0.5438 0.5832 0.6217 0.6591 0.6950 0.7291 0.7611 0.7910 0.8186 0.8438 0.8665 0.8869 0.9049 0.9207 0.9345 0.9463 0.9564 0.9649 0.9719 0.9778 0.9826 0.9864 0.9896 0.9920 0.9940 0.9955 0.9966 0.9975 0.9982 0.9987
0.02 0.5080 0.5478 0.5871 0.6255 0.6628 0.6985 0.7324 0.7642 0.7939 0.8212 0.8461 0.8686 0.8888 0.9066 0.9222 0.9357 0.9474 0.9573 0.9656 0.9726 0.9783 0.9830 0.9868 0.9898 0.9922 0.9941 0.9956 0.9967 0.9976 0.9982 0.9987
0.03 0.5120 0.5517 0.5910 0.6293 0.6664 0.7019 0.7357 0.7673 0.7967 0.8238 0.8485 0.8708 0.8907 0.9082 0.9236 0.9370 0.9484 0.9582 0.9664 0.9732 0.9788 0.9834 0.9871 0.9901 0.9925 0.9943 0.9957 0.9968 0.9977 0.9983 0.9988
0.04 0.5160 0.5557 0.5948 0.6331 0.6700 0.7054 0.7389 0.7704 0.7995 0.8264 0.8508 0.8729 0.8925 0.9099 0.9251 0.9382 0.9495 0.9591 0.9671 0.9738 0.9793 0.9838 0.9875 0.9904 0.9927 0.9945 0.9959 0.9969 0.9977 0.9984 0.9988
0.05 0.5199 0.5596 0.5987 0.6368 0.6736 0.7088 0.7422 0.7734 0.8023 0.8289 0.8531 0.8749 0.8944 0.9115 0.9265 0.9394 0.9505 0.9599 0.9678 0.9744 0.9798 0.9842 0.9878 0.9906 0.9929 0.9946 0.9960 0.9970 0.9978 0.9984 0.9989
0.06 0.5239 0.5636 0.6026 0.6406 0.6772 0.7123 0.7454 0.7764 0.8051 0.8315 0.8554 0.8770 0.8962 0.9131 0.9279 0.9406 0.9515 0.9608 0.9686 0.9750 0.9803 0.9846 0.9881 0.9909 0.9931 0.9948 0.9961 0.9971 0.9979 0.9985 0.9989
0.07 0.5279 0.5675 0.6064 0.6443 0.6808 0.7157 0.7486 0.7794 0.8078 0.8340 0.8577 0.8790 0.8980 0.9147 0.9292 0.9418 0.9525 0.9616 0.9693 0.9756 0.9808 0.9850 0.9884 0.9911 0.9932 0.9949 0.9962 0.9972 0.9979 0.9985 0.9989
0.08 0.5319 0.5714 0.6103 0.6480 0.6844 0.7190 0.7517 0.7823 0.8106 0.8365 0.8599 0.8810 0.8997 0.9162 0.9306 0.9429 0.9535 0.9625 0.9699 0.9761 0.9812 0.9854 0.9887 0.9913 0.9934 0.9951 0.9963 0.9973 0.9980 0.9986 0.9990
0.09 0.5359 0.5753 0.6141 0.6517 0.6879 0.7224 0.7549 0.7852 0.8133 0.8389 0.8621 0.8830 0.9015 0.9177 0.9319 0.9441 0.9545 0.9633 0.9706 0.9767 0.9817 0.9857 0.9890 0.9916 0.9936 0.9952 0.9964 0.9974 0.9981 0.9986 0.9990
1 - .9826 = .0174
27
The Z-test
The question Does a sample differ from a specific population with known mean and standard deviation? Are the anxiety levels (MAS scores) of UW PSY225 students different from the general population? What would the potential predictions look like (verbally and graphically)? [Think about this for a moment]
28
Population Means are the Same [Same population]
0.14 0.12 0.1 Probability 0.08 0.06 0.04 0.02 0 -20 -15 -10 -5 0 5 10 15 Score 20 25 30 35 40 45 50
29
PSY225 Students are More Anxious
0.14 0.12 0.1 Probability 0.08 0.06 0.04 0.02 0 -20 -15 -10 -5 0 5 10 15 Score 20 25 30 35 40 45 50
30
PSY225 Students are Less Anxious
0.14 0.12 0.1 Probability 0.08 0.06 0.04 0.02 0 -20 -15 -10 -5 0 5 10 15 Score 20 25 30 35 40 45 50
31
PSY225 Students are "Different" (Non-directional)
0.14 0.12 0.1 Probability 0.08 0.06 0.04 0.02 0 -20 -15 -10 -5 0 5 10 15 Score 20 25 30 35 40 45 50
Null Hypothesis:
Non-directional Hypothesis:
0.14 0.12 0.1 Probability 0.08 0.06 0.04 0.02 0 -20 -15 -10 -5 0 5 10 15 20 25 30 35 40 45 50
32
The Z-test
The question [General] Does a specific sample differ from a specific population with known mean and standard deviation? [Specific] Are the anxiety scores (MAS scores) of UW PSY225 students different from the general population of people? 1. Through extensive testing, I know that the general population has a mean of 15 and a SD of 10 on the MAS 2. We gather a sample of UW PSY225 students (N=73) 3. Calculate the mean MAS of the sample (mean =18.6)
33
The Z-test
The sample mean for the UW students (18.6) is different from the mean of the population of non-students (15). We know that the sample mean is an unbiased estimator of the population mean. Can we conclude that that the UW students come from a different population with a higher level of anxiety?
34
The Z-test
How can we determine how probable a particular sample mean is if we assume it came from the non-student distribution?
35
The Z-test
We now know that: UW sample's mean is 3.6 MAS points above the non-student population mean and the SD of sample means (N=73) from non-student population (i.e., the standard error/sampling error) is 1.2 How do we figure out how probable that a sample with this mean came from non-student population?
36
The Z-test
How likely is it to have a score that is 3 standard deviations (above or below) or more from the mean?
What is this the probability of?
What is this in statistics?
37
The Z-test
You have developed your first statistical test!! The Z-test
Z = Meansample Meanpop SDpop/sqrt(N) You have also derived the basic logic of Null Hypothesis Significance Testing (NHST)!
38
The Z-test
The distribution of sample means (in blue; N=73) from the non-college population of scores (in yellow)
0.4 0.14 0.35 0.12 0.3 0.1 Probability 0.25 0.08 0.2 0.06 0.15 0.04 0.1 0.02 0.05 0 -20 -15 -10 -5 0 5 10 15 Score 20 25 30 35 40 45 50
39
Null Hypothesis Significance Testing (NHST)
1. Divide reality into two non-overlapping possibilities. (Null hypothesis & Alternate hypothesis). 1. Assume that the Null hypothesis is true. 1. Collect data. 1. Calculate the probability (p-value) of obtaining your data (or more extreme data) given your assumption (e.g., that sample came from the known population and only differences should be result of sampling error). 1. Compare probability to some cut-off value (alpha level). 1. (a) If data is less probable than cut-off value, reject null hypothesis in favor of alternate hypothesis. 1. (b) If data is not less probable, fail to reject Null hypothesis.
40
Key Concepts in NHST
Null Hypothesis Assumption that there is no difference between groups or no relationship between variables. Exact form of the null (Ho) varies based on the specific inferential test. Examples: UW students are not different from general population with respect to anxiety (z-test if we know M/SD for one population) There is no relationship between frequency/quantity of alcohol consumption and class performance (correlation) There is no difference between the mean cleanliness of men and women (t-test if we don't know the descriptive stats for either population) 41
Key Concepts in NHST
Alternative hypothesis: There is a difference between groups or a relationship between variables. Alternative hypotheses are either nondirectional (two-tailed) or directional (one-tailed).
Examples UW students mean anxiety is different from mean anxiety in general population (non-directional) UW students mean anxiety is higher than general population mean anxiety (directional) Quantity/frequency of drinking is related to class performance (nondirectional) As quantity/frequency of drinking increases, performance decreases (directional) Men and women are different in mean cleanliness (non-directional) Men are more clean than women (directional)
42
UW Students are More Anxious
0.14 0.12 0.1 Probability 0.08 0.06 0.04 0.02 0 -20 -15 -10 -5 0 5 10 15 Score 20 25 30 35 40 45 50
43
Directional Alternative Hypothesis
0.4 0.35 0.3 0.25 0.2 0.15 0.1 0.05 0 8 9 10 11 12 13 14 15 Score 16 17 18 19 20 21 22
Probability
44
UW Students have different Anxiety (more or less)
0.14 0.12 0.1 Probability 0.08 0.06 0.04 0.02 0 -20 -15 -10 -5 0 5 10 15 Score 20 25 30 35 40 45 50
45
Non-Directional Alternative Hypothesis
0.4 0.35 0.3 0.25 0.2 0.15 0.1 0.05 0 8 9 10 11 12 13 14 15 Score 16 17 18 19 20 21 22
Probability
46
Key Concepts in NHST
p-value The probability of obtaining your data (or data more extreme) if the null hypothesis is true. Alpha level A preset cut-off to which the p-value is compared. Statistical significance p-value is less than alpha and null hypothesis is rejected. Type I error Rejecting null hypothesis when it is true. (a.k.a. False alarm) Type II error Failing to reject null hypothesis when it is false. (a.k.a. Miss)
47
Errors in NHST
Reality H0 True Reject H0 Decision H0 False
Type I error False Alarm (alpha) Correct Decision
Correct Decision Type II error Miss (1-power) (beta)
FTR H0
48
Alpha and Type I Errors
0.4 0.35 0.3 0.25 0.2 0.15 0.1 0.05 0 8 9 10 11 12 13 14 15 Score 16 17 18 19 20 21 22
Probability
Alpha = .05
Alpha = .01
49
Statistical Power
Power The probability of rejecting the null hypothesis if it is false Flip side of power is probability of "miss" Your ability to detect an effect if it exists 80% power is the rule of thumb minimum power level In other words, if a effect existed, you would find it (reject the null) 80 experiments out of 100
50
Factors that Affect Statistical Power
1. The statistical test 1. The effect size (d, f, r) 1. The sample size 1. Alpha
51
Statistical Power: The Z-test
Z-test is a simpler version of the t-test Only have to consider one population (and the sampling distribution associated with it) and determine if your sample is from it
Z = Meansample Meanpop SDpop/sqrt(N)
0.4 0.35 0.3 0.25 0.2 0.15 0.1 0.05 0 8 9 10 11 12 13 14 15 Score 16 17 18 19 20 21 22
Given what you now know (and viewing the formula above), what factors affect the power of the Z-test?
Probability
52
Power and Type II Errors: Original Data
Non-Student: Mean=15; SD=10; N=73 (SE=1.2) UW: Population Mean=19 Z = Meansample Meanpop SDpop/sqrt(N)
0.4 0.35 0.3 Probability 0.25 0.2 0.15 0.1 0.05 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Score
53
Calculating Critical Value
If we used a one-tailed (directional) test [predicting UW students more anxious], what is the minimum UW sample mean that would result in rejecting the null at alpha = .05?
General pop: Mean=15; SD=10; N=73 (SE=1.2) Z = Meansample Meanpop SDpop/sqrt(N)
What z-score would result in rejecting the null (remember it is a directional alternative)? How do you convert that z-score into a raw MAS score?
54
Power and Type II Errors: Original Data
Non-Student: Mean=15; SD=10; N=73 (SE=1.2) UW: Population Mean=19 Z = Meansample Meanpop SDpop/sqrt(N)
0.4 0.35 0.3 Probability 0.25 0.2 0.15 0.1 0.05 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Score
55
Power and Type II Errors: > Mean Diff
Non-Student: Mean=15; SD=10; N=73 (SE=1.2) UW: Population Mean=19 UW: Population Mean=21
0.4 0.35 0.3 Probability 0.25 0.2 0.15 0.1 0.05 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Score
Z = Meansample Meanpop SDpop/sqrt(N)
56
Power and Type II Errors: Original Data
Non-Student: Mean=15; SD=10; N=73 (SE=1.2) UW: Population Mean=19 Z = Meansample Meanpop SDpop/sqrt(N)
0.4 0.35 0.3 Probability 0.25 0.2 0.15 0.1 0.05 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 21 20 22 23 24 25 26 27 28 29 30 Score
57
Power and Type II Errors: < SD
Z = Meansample Meanpop SDpop/sqrt(N)
Non-Student: Mean=15; SD=5; N=73 (SE=0.58) UW: Population Mean=19 wo things changed that increased power?
0.60 0.50 0.40 Probability 0.30 0.20 0.10 0.00
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
Score
58
Power and Type II Errors: Original Data
Non-Student: Mean=15; SD=10; N=73 (SE=1.2) UW: Population Mean=19 Z = Meansample Meanpop SDpop/sqrt(N)
0.4 0.35 0.3 Probability 0.25 0.2 0.15 0.1 0.05 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Score
59
Power and Type II Errors: > N
Non-Student: Mean=15; SD=10; N=150 (SE=0.81) Z = Meansample Meanpop SDpop/sqrt(N) UW: Population Mean=19
0.50 0.45 0.40 0.35 Probability 0.30 0.25 0.20 0.15 0.10 0.05 0.00 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Score
60
Power and Type II Errors: Decrease Alpha
Non-Student: Mean=15; SD=10; N=73 (SE=1.2) UW: Population Mean=19 Z = Meansample Meanpop SDpop/sqrt(N)
With alpha=.05: z> 1.65 with sample mean > 16.9 With alpha=.01: z> 2.33 with sample mean > 17.7
0.4 0.35 0.3 Probability 0.25 0.2 0.15 0.1 0.05 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Score
61
Statistical Power: z-test vs. t-test
z-Test One sample taken to determine if sample is from known population (or not) One sampling distribution (assume sample is from known population) Not incredibly useful b/c we usually don't have known info about population. Nice model though
z = Meansample Meanpop SDpop/sqrt(N) t-Test Two samples taken to determine if they come from same population (or not) One sampling distribution (which is the difference between these two samples assuming same population) SD of sampling distribution (i.e., SE) is different t= Mean1 Mean2 SQRT((S21/N1) + (S22/N2))
62
When is power analysis important?
1. A priori for sample size planning An a priori power analysis should always be conducted prior to conducting an experiment to determine what an adequate number of participants will be to detect an effect size "of interest" What are the advantages vs. disadvantages of increasing sample size?
63
When is power analysis important?
How does the probability of type II error change as sample size increases?
How does the probability of type I error change as sample size increases?
64
When is power analysis important?
When else would it be important to conduct a power analysis?
65
When is power analysis important?
2. Post hoc if failed to reject null If you failed to reject the null, a power analysis should be conducted to determine the probability that this failure is "a miss". Here you should indicate the smallest effect that could have been detected with "adequate" power
66
Statistical Power: How to Calculate Sample Size
Factors that affect power are:
Therefore, if you set the effect size, and alpha and indicate that you want 80% power, one specific sample size will provide that power
ow do we set alpha?
67
Statistical Power: How to Calculate Sample Size
How do we quantify effect size? It depends on the statistical test For t-test, the effect size index is d d= Mean1 Mean2 (SD1 + SD2) / 2
ow do we determine effect size parameters for calculating d?
e want to determine what the minimum effect size of interest Estimate smallest mean difference that is important a. Get SD from pilot data b. Get SD from other studies in literature Determine overall d using rule of thumb effect size
68
Statistical Power: How to Calculate Sample Size
Rules of thumb for effect size (d) Small effect = .20 Medium effect = .50 Large effect = .80 What does an effect size of .50 indicate?
1 2
d=
Mean Mean
1 2
(SD + SD ) / 2
69
Effect Size: d = .20
0.14 0.12 0.10 Probability 0.08 0.06 0.04 0.02 0.00 -10 -8 -6 -4 -2 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 Score
Raw score distribution
70
Effect Size: d = .50
0.14 0.12 0.10 Probability 0.08 0.06 0.04 0.02 0.00 -10 -8 -6 -4 -2 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 Score
Raw score distribution
71
Effect Size: d = .80
0.14 0.12 0.10 Probability 0.08 0.06 0.04 0.02 0.00 -10 -8 -6 -4 -2 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 Score
Raw score distribution
72
Effect Size: d = .20
0.30 0.25 0.20 Probability 0.15 0.10 0.05 0.00 -10 -8 -6 -4 -2 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 Score
Sampling distribution of the mean (n=25 per sample)
73
Effect Size: d = .50
0.30 0.25 0.20 Probability 0.15 0.10 0.05 0.00 -10 -8 -6 -4 -2 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 Score
Sampling distribution of the mean (n=25 per sample)
74
Effect Size: d = .80
0.30 0.25 0.20 Probability 0.15 0.10 0.05 0.00 -10 -8 -6 -4 -2 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 Score
Sampling distribution of the mean (n=25 per sample)
75
Sample Size Planning
How many participants does it take to have adequate power to detect differences between Floridians and Non-Floridians on cold tolerance? Want to detect a small effect (d=.2) at a two tailed alpha of . 01 with 80% power
What if we changed the alpha to .05 and were interested in only detecting a medium effect?
76
Validity of Research Design
Statistical Validity The results are due to systematic factor rather than chance variation between samples. You want maximum POWER! Construct Validity The measurement and/or manipulations of observed variables are related to the underlying constructs of interest (both IV and DV) External Validity The results are generalizable to other subjects, conditions, times, and places. Internal Validity The IV is causally related to the DV.
77
Development of Differential/Experimental Study
1. Identify Question 1. Select Sample, Sampling Method, & Sample Size 2. Select groups or manipulation (Operationalize IV) 3. Determine design A. Differential vs. Experimental design B. Between vs. Within design 4. Operationalize DV 5. Standardize procedure 6. Assign subjects and collect data 7. Select and conduct appropriate statistical analyses
78
Steps in Research Hypothesis Formation
Initial ideas (Often Vague and General)
Initial Observations
Literature Review
(Theory, other studies)
Statement of Problem
(Should be theory driven)
Research Hypothesis (a specific deductive prediction)
79
Development of Differential/Experimental Study
1. Identify Question 1. Select Sample, Sampling Method, & Sample Size 1. Select groups or manipulation (Operationalize IV) 2. Determine design A. Differential vs. Experimental design B. Between vs. Within design 3. Operationalize DV 4. Standardize procedure 5. Assign subjects and collect data 6. Select and conduct appropriate statistical analyses
80
Sample/Population Characteristics
1. Sample should reflect the characteristics of the population to whom you want to generalize your results. 1. The more diverse the population, the broader the population that your results apply to. Is diversity (heterogeneity) in your sample always good?
81
Sample/Population Characteristics
Theory may also have implications for the selection of the populations to study. Depending on the proposed mechanism through which the IV affects DV, may have different predictions of effects based on population Differences in attention among anxious and non-anxious individuals Example of alcohol effects among low executive function individuals
82
Sample/Population Characteristics
Who should our sample be? World Class Athletes, couch potatoes, all levels of athletic experience Men, women, mixed gender? Children, young adults, adults, geriatrics? Need to develop formal Inclusion and Exclusion criteria Start with inclusion criteria and then specify any exclusions within the group defined by inclusion criteria
Do we need to exclude any individuals and why?
83
Sampling Method
Random Sampling Random sampling involves randomly selecting participants from the population of interest to participate in your study. This method, if done correctly, guarantees that your sample will reflect the characteristics of your target population.
Ad hoc (Convenience) Sampling Convenience sampling involves relying on participants who are readily accessible. Often it involves volunteers and often the participants contact the researcher.
84
Sampling Method
Is random sampling always better?
85
Sampling Method
Random Sampling vs. Random Assignment Random sampling is randomly selecting participants to include in your sample that will participate in your study. Random sampling will affect your external validity
Random assignment is randomly placing the sample participants in the various groups or levels of your manipulated IV. Random assignment is the foundation of an experimental design and is a key factor in establishing internal validity.
Random sampling is NOT necessary for an internally valid study 86
Sample Size
Why are larger samples better?
What is the downside to large samples? Should you always get the largest sample you can afford?
87
Development of Differential/Experimental Study
1. Identify Question 2. Select Sample, Sampling Method, & Sample Size 1. Select groups or manipulation (Operationalize IV) 1. Determine design A. Differential vs. Experimental design B. Between vs. Within design 2. Operationalize DV 3. Standardize procedure 4. Assign subjects and collect data 5. Select and conduct appropriate statistical analyses
88
Two group designs: The groups
Experimental Group Group of subjects used in either differential or experimental research that are assigned to (or exhibit) a specified level of the independent variable. The experimental group(s) is/are usually contrasted with a control group
Control Group A group of subjects used in either differential or experimental research that serve as a basis of comparisons for other (experimental) groups. The ideal control group is similar to the experimental group on all variables except the variable that defines the group (i.e., the independent variable)
89
Placebo vs. Control Group
What is a placebo effect?
How do we control for placebo effects?
90
Single and Double Blind Experiments
Using a placebo control group is one type of a more general control procedure called a single blind design To be blind means to not know information about the IV level or group. Specifically, In a single blind experiment, the participant does not know what level of the IV s/he is assigned In a double blind experiment, neither the participant nor the RAs (who interact with the participant) know what condition the participant is in.
When are blinding procedures most important and are they ever bad?
91
Extreme Group vs. Median Split Group
Imagine you are investigating the effect of trait anxiety on alcohol use. You believe that one reason that people drink is to reduce anxiety. From this theory, it would be expected that level of trait anxiety should be related to drinking frequency. How do you investigate this?
How do you determine group membership?
92
Extreme Group vs. Median Split Group
With median split assignment, you will: Determine scores for all potential participants Determine the median score among participants Individuals above median go in one group (hi anxious) Individuals below median go in other group (lo anxious) With extreme group assignment, you will: Determine scores for all potential participants Determine upper and lower groups (typically top and bottom 25%) Individuals above 75%ile go in one group (hi anxious) Individuals below 25%ile go in other group (lo anxious)
93
Extreme Group vs. Median Split Group
Which is better?
94
Development of Differential/Experimental Study
1. Identify Question 2. Select Sample, Sampling Method, & Sample Size 3. Select groups or manipulation (Operationalize IV) 1. Determine design A. Differential vs. Experimental design B. Between vs. Within design 1. Operationalize DV 2. Standardize procedure 3. Assign subjects and collect data 4. Select and conduct appropriate statistical analyses
95
Design: Differential vs. Experimental
Differential Design IV group assignment is based on pre-exisiting differences. The researcher does not have control over the IV. IV is a "grouping" variable.
Experimental Design The researcher has direct control over the IV. The researcher can (and does) randomly assign participants to the various groups or levels of the IV.
96
Design: Differential vs. Experimental
Which is better, experimental or differential designs?
Advantages of Experimental Design Experimental designs allow stronger statements about causality Random assignment controls for "other differences" between the groups. Should still control "important&q...