Statistical Applications:
Robert Pred, Ph.D., page 1 of 11
Drawing Conclusions:
Statistical Power and Errors in Hypothesis Testing
Overview of Decision Making in Research:
In the conduct of an experimental study, we as researchers must acknowledge that at best, we draw conclusions from
our results under conditions of "uncertainty." That is, when we reach statistical conclusions, by either rejecting the
null, or by accepting the null, we may not always be led by our "data" to make the "appropriate" decision. There are
four possible scenarios, each defined by their likelihood, to consider whenever we draw conclusions from our data.
These four probabilistically determined outcomes of research are presented below.
Statistical Power
:
Correctly Rejecting the Null
The likelihood of being led by our sample data to reject the null, when in fact, the null hypothesis should be rejected,
represents a correct decision, and this is known as "statistical power." Statistical power is defined as the likelihood of
detecting a difference between a sample statistic and a hypothesized population parameter, when in reality, such a
difference really exists. Power is typically expressed as the "likelihood" (i.e., "the probability") of rejecting the null
hypothesis, when in fact the null hypothesis is really false. When planning a research study, and certainly before data
collection begins, efforts should be made to estimate the statistical power of your study. Entire reference books are
available that are devoted entirely to the calculation of power for the conduct of statistical tests.
1
Type I Error (Alpha level)
:
Incorrectly Rejecting the Null
The likelihood of being led by our sample data to reject the null, when in fact the null should be accepted represents a
decision error known as a Type I error.
1
For example: Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Erlbaum.
Copyright 1999 by Robert Pred, Ph.D.; reproduction by any means without permission is prohibited.