Inference
More Statistical discussion on material covered in D&B Chapters 8 and 9
Weights of Bears
The health of the bear population in Yellowstone National Park is monitored by periodic measurements taken from anesthetized bears. A sample of 54 bears has a mean weight of 182.9 pounds. The sample standard deviation is estimated to be 121.8 pounds. Use a 0.05 significance level to test the claim that the population mean of all such bear weights is different than 200 pounds. Bear.t.test.ssc
Most statisticians are familiar with conversations that start:
Q: What is the purpose of your analysis? A: I want to do a significance test. Q: No, I mean what is the overall objective? A (with puzzled look): I want to know if my results are significant. And so on...." (Chatfield 1991)
Hypothesis Testing
Formulate a hypothesis Conduct an experiment Test for significance Dont stop here! Draw conclusions Make predictions Ask more questions
One Tailed vs. Two Tailed z Tests
0.4
P(Z)
0.0
0.1
0.2
qnorm(.975) 1.959964 qnorm(.95) 1.644854 qnorm(.05) 1.644854
0.3 -3
-2
-1
0 Z
1
2
3
One Tailed vs. Two Tailed t Tests
qnorm(0.975) 1.959964 qt(0.975,df=300) 1.967903 qt(0.975,df=30) 2.042272 qt(0.975,df=3) 3.182446
0.4 P(Z) 0.0 -3 0.1 0.2 0.3
-2
-1
0 Z
1
2
3
Difference in the Mean vs. Difference in the Variance
More on this later
0.10
X = 0:20 Px = dchisq(x,10) plot(x,px, type='l',lwd=3)
px 0.0 0 0.02 0.04
0.06
0.08
5
10 x
15
20
Normal vs.Other Distributions
x = 0:20 px = dpois(x,10) plot(x,px, type='b',lwd=3)
0.12 px 0.0 0 0.02 0.04 0.06 0.08 0.10
5
10 x
15
20
The Standardized Normal Distribution
f ( z | z = 0, z z= x x
2
1 = 1) = e 2
z2 2
x
P(Z)
0.0 -3
0.1
0.2
0.3
0.4
-2
-1
0 Z
1
2
3
Using the Standard Normal in Hypothesis Testing
f ( z | z = 0, z
2
1 = 1) = e 2
z2 2
x0 0 Hypothetical z0 = s x0 x1 o Actual z1 = s x1
P(Z) 0.3 0.0 -3 0.1 0.2 0.4
-2
-1
0 Z
1
2
3
Probability Distribution of the Hypothetical Mean
0.4
Hypothetical x0 0 z0 = s x0
P(Z)
0.0 -3
0.1
0.2
0.3
-2
-1
0 Z
1
2
3
Distribution of Mean
0.4
Hypothetical x0 0 z0 = s x0
0.3
95% CI
Probability
0.0 -3
0.1
0.2
-2
-1
0 x Z
1
2
3
Hypothesis Testing
How confident are we that what we are looking at has not occurred by chance?
Hypothesis Testing
Type I Error:
Probability that you think you have a difference when in fact it there is no difference.
Type II Error:
Probability that you think there is no difference when in fact there is a difference.
Accept H0
H0 True
Reject H0
H1 True
Probability of false reject if H0 is true
H0 True
H1 True
^ Critical Value
Probability of false accept if H1 is true
H0 True
H1 True
^ Critical Value
Power of the Test to Detect Significance
H0 True
H1 True
1
P-value = Probability of obtaining a mean as extreme or more extreme than the sample mean under the null hypothesis
H0 True
P-value = 0.05 P-value = 0.01
Control and Case (Treatment)
H0 True
H1 True
Effect of Sample Size
H0 True
H1 True
Form of Alternative Hypotheses
H0 True
Hypothesis Testing Expanded
Formulate hypotheses (Ask questions) Experiment (Make observations) Examine significance
Statistical: Was it measurable? Biological: Is it important?
Draw inferences (What does it mean?) Make predictions Ask more questions
Implications
Statistical significance reflects ones ability to measure a difference, and does not necessarily denote biological relevance
Implications
Hypothesis formulation and testing is a useful component of this logical process In that sense, it adds (sometimes needed) logical structure to thought But, it may not adequately reflect problem complexity Our questions need to be relevant, testable and hopefully interesting! use So hypothesis formulation to clarify, but dont over simplify!
Implications
Get to know these terms: alpha: reject when should accept beta: accept when should reject power: information against the null hypothesis p-value: probability of getting a test statistic that is at least as extreme as the one representing the sample data, if null is true significant: statistically measurable These terms are the statistical vocabulary of the scientific realm However, they are means, not ends
Implications
It may be more important to keep in mind:
What questions are we attempting to answer? How should observations be appropriately gathered? What conclusions can we draw? What predictions can we make? Are there further questions we should ask?
Risk and Probability
Space Shuttle Failure
http://www.fotuva.org/feynman/ http://history.nasa.gov/rogersrep/v2appf.htm
Challender Expert Speaks to Students by Katie Schlegel for THE POST
Challenger Disaster Presidential Commission The commission includes: Astronauts Sally Ride and Neil Armstrong Pilot Chuck Yeager Physicist Richard Feynman Objectives: Determine the cause of the accident Helping prevent another Problem discovered:
A flaw in the decision-making process at NASA
Challender Expert Speaks to Students by Katie Schlegel for THE POST
Engineers knew of the O-ring problem But it was just one of 131 "criticality one" components in the shuttle. These O-rings were parts of the shuttle known to have problems and constantly were reworked. The engineers also knew the cold could affect the launch, but they knew much could be lost by delaying it. There would be a loss in federal funding and a loss of time and momentum in the race to build a space station.
Challender Expert Speaks to Students by Katie Schlegel for THE POST
This whole decision-making process has become a classic case in group think and the wrong way to approach the decision-making process. They didn't think this was going to be the outcome. They thought they could launch again and get away with it. Unfortunately, they ran out of luck, and so did seven people.
NASA Diagram of O-Ring Failure
Tufte Diagram of O-Ring Failure
Richard Feynman, Physicist California Institute of Technology (Caltech)
The next day following the closed meeting, an open, televised meeting was scheduled. Feynman started his morning with a trip to the hardware store and got some tools. As the meeting convened, he asked for ice water. He took some O-ring pieces he had in his pocket, squeezed them in a C-clamp, and put them into the glass of ice water. He pressed the button for the microphone, and announced the results of his experiment: I discovered that when you undo the clamp, the rubber doesnt spring back. In other words, for more than a few seconds, there is no resilience in this particular material when it is at a temperature of 32 degrees. I believe that has some significance for our problem.
Richard Feynman, Physicist California Institute of Technology (Caltech)
Richard Feynman addresses the possible shortcomings of NASA Management in the 1986 official report of the Challenger Disaster Presidential Commission Deceptive to characterize a Space Shuttle as if it had attained the safety of an ordinary airliner
Richard Feynman, Physicist California Institute of Technology (Caltech)
NASA Management was claiming that the odds of losing a Shuttle were 1 in a 100,000 Equivalent to launching a Shuttle every day for a hundred years and only losing one. Line Engineers: Odds are likely to be more on the order of losing 1 in a 100 flights. We have now lost two in 113 flights (or 1 in 56.5.) Feynman: For a successful technology, reality must take precedence over public relations, for nature cannot be fooled