the Managing Producer's or the Consumer's Risk The logic behind a statistical test of hypothesis is similar to the following logic. Draw two lines on a paper and determine whether they are of different lengths. You compare them and say, "Well, certainly they are not equal. Therefore they must be of different lengths. By rejecting equality, that is, the null hypothesis, you assert that there is a difference. The power of a statistical test is best explained by the overview of the Type I and Type II errors. The following matrix shows the basic representation of these errors.
The Type-I and Type-II Errors As indicated in the above matrix a Type-I error occurs when, based on your data, you reject the null hypothesis when in fact it is true. The probability of a type-I error is the level of significance of the test of hypothesis and is denoted by . Type-I error is often called the producer's risk that consumers reject a good product or service indicated by the null hypothesis. That is, a producer introduces a good product, in doing so, he or she take a risk that consumer will reject it. A type II error occurs when you do not reject the null hypothesis when it is in fact false. The probability of a type-II error is denoted by . The quantity 1 - is known as the Power of a Test. A Type-II error can be evaluated for any specific alternative hypotheses stated in the form "Not Equal to" as a competing hypothesis. Type-II error is often called the consumer's risk for not rejecting possibly a worthless product or service indicated by the null hypothesis. Students often raise questions, such as what are the 'right' confidence intervals, and why do most people use the 95% level? The answer is that the decision-maker must consider both the Type I and II errors and work out the best tradeoff. Ideally one wishes to reduce the probability of making these types of error; however, for a fixed sample size, we cannot reduce one type of error without at the same time increasing the probability of another type of error. Nevertheless, to reduce the probabilities of both types of error simultaneously is to increase the sample size. That is, by having more information one makes a better decision. The following example highlights this concept. A electronics firm, Big Z, manufactures and sells a component part to a radio manufacturer, Big Y. Big Z consistently maintain a component part failure rate of 10% per 1000 parts Here produced. Big Z is the producer and Big Y is the consumer. Big Y, for
reasons of practicality, will test sample of 10 parts out of lots of 1000. Big Y will adopt one of two rules regarding lot acceptance: Rule 1: Accept lots with one or fewer defectives; therefore, a lot has either 0 defective or 1 defective. Rule 2: Accept lots with two or fewer defectives; therefore, a lot has either 0,1, or 2 defective(s). On the basis of the binomial distribution, the P(0 or 1) is 0.7367. This means that, with a defective rate of 0.10, the Big Y will accept 74% of tested lots and will reject 26% of the lots even though they are good lots. The 26% is the producer's risk or the level. This level is analogous to a Type I error -- rejecting a true null. Or, in other words, rejecting a good lot. In this example, for illustration purposes, the lot represents a null hypothesis. The rejected lot goes back to the producer; hence, producer's risk. If Big Y is to take rule 2, then the producer's risk decreases. The P(0 or, or 1, or 2) is 0.9298 therefore, Big Y will accept 93% of all tested lots, and 7% will be rejected, even though the lot is acceptable. The primary reason for this is that, although the probability of defective is 0.10, the Big Y through rule 2 allows for a higher defective acceptance rate. Big Y increases its own risk (consumer's risk), as stated previously. Making Good Decision: Given that there is a relevant profit (which could be negative) for the outcome of your decision, and a prior probability (before testing) for the null hypothesis to be true, the objective is to make a good decision. Let us denote the profits for each cell in the decision table as $a, $b, $c and $d (columnwise), respectively. The expectation of profit is [a + (1-)b], and + [(1-)c + d], depending whether the null is true. Now having a prior (i.e., before testing) subjective probability of p that the null is true, then the expected profit of your decision is: Net Profit = [a + (1-)b]p + [(1-)c + d](1-p) - Sampling cost A good decision makes this profit as large as possible. To this end, we must suitably choose the sample size and all other factors in the above profit function. Note that, since we are using a subjective probability expressing the strength of belief assessment of the truthfulness of the null hypothesis, it is called a Bayesian Approach to statistical decision making, which is a standard approach in decision theory.