Coursehero >> Pennsylvania >> Penn State >> STAT 504

Course Hero has millions of student submitted documents similar to the one below including study guides, homework solutions, papers, exam answer keys and textbook solutions.

504, Stat Lecture 4 1 ' $ What if the respondents in the survey had three choices: (1) I feel optimistic, (2) I don't feel optimistic, (3) I am not sure? What would you consider as an appropriate sampling distribution? & % Stat 504, Lecture 4 2 ' $ The Multinomial Distribution Origins The multinomial distribution arises from an extension of the binomial experiment to situations where each trial has k 2 possible outcomes. Suppose that we have an experiment with n independent trials, where each trial produces exactly one of the events E1 , E2 , . . . , Ek (i.e. these events are mutually exclusive and collectively exhaustive), and on each trial, Ej occurs with probability j , j = 1, 2, . . . , k. Notice that 1 + 2 + + k = 1. & % Stat 504, Lecture 4 3 ' Define the random variables X1 X2 = = . . . = number of trials in which E1 occurs, number of trials in which E2 occurs, $ Xk number of trials in which Ek occurs. Then X = (X1 , X2 , . . . , Xk ) is said to have a multinomial distribution with index n and parameter = (1 , 2 , . . . , k ). In most problems, n is regarded as fixed and known. The individual components of a multinomial random vector are binomial, X1 X2 . . . Bin(n, 1 ), Bin(n, 2 ), Xk Bin(n, k ). However, the components are not independent. Even though the individual Xj 's are random, their sum X1 + X2 + + Xk = n & is fixed. Therefore, the Xj 's are negatively correlated. % Stat 504, Lecture 4 4 ' $ Notation. If X = (X1 , X2 , . . . , Xk ) is multinomially distributed with index n and parameter = (1 , 2 , . . . , k ), then we will write X Mult(n, ). Distribution function. The probability that X = (X1 , . . . , Xk ) takes a particular value x = (x1 , . . . , xk ) is f (x) = n! x x x 1 1 2 2 k k . x1 ! x2 ! xk ! The possible values of X are the set of x-vectors such that each xj {0, 1, . . . , n} and x1 + + xk = n. & % Stat 504, Lecture 4 5 ' $ Example. Suppose that the racial/ethnic distribution in a large city is given by this table: Black 20% Hispanic 15% Other 65% Suppose that a jury of twelve members is chosen from this city in such a way that each resident has an equal probability of being selected independently of every other resident. & % Stat 504, Lecture 4 6 ' Let's find probability that the jury contains three Black, two Hispanic, and seven Other members; four Black and eight Other members; at most one Black member. To solve this problem, let X = (X1 , X2 , X3 ) where X1 = number of Black members, X2 = number of Hispanic members, and X3 = number of Other members. Then X has a multinomial distribution with parameters n = 12 and = (.20, .15, .65). The answer to the first part is P (X1 = 3, X2 = 2, X3 = 7) = = = n! x x x 1 1 2 2 3 3 x1 ! x2 ! x3 ! 12! (.20)3 (.15)2 (.65)7 3! 2! 7! 0.0699. $ The answer to the second part is P (X1 = 4, X2 = 0, X3 = 8) = = 12! (.20)4 (.15)0 (.65)8 4! 0! 8! 0.0252. & % Stat 504, Lecture 4 7 ' $ For the last part, note that "at most one Black member" means X1 = 0 or X1 = 1. X1 is a binomial random variable with n = 12 and p = 1 = .2. Using the binomial probability distribution, P (X1 = 0) = = and P (X1 = 1) = = Therefore, the answer is P (X1 = 0) + P (X1 = 1) = 0.0687 + 0.2061 = 0.2748. 12! (.2)1 (.8)11 1! 11! 0.2061. 12! (.2)0 (.8)12 0! 12! 0.0687 & % Stat 504, Lecture 4 8 ' Moments. Many of the elementary properties of the multinomial can be derived by decomposing X as the sum of iid random vectors X = Y1 + Y2 + Yn , where each Yi Mult(1, ). In this decomposition, Yi represents the outcome of the ith trial; it's a vector with a 1 in position j if Ej occurred on that trial and 0's in all other positions. The elements of Yi are correlated Bernoulli's. For example, with k = 2 possible outcomes on each trial, the possible values of Yi are (1, 0) (0, 1) with probability with probability 1 , 2 = 1 - 1 . $ Because the individual elements of Yi are Bernoulli, the mean of Yi is = (1 , 2 ), and its covariance matrix is 2 3 -1 2 (1 - 1 ) 4 1 5. -1 2 2 (1 - 2 ) (To see that the off-diagonal elements are -1 2 , one can use the definition of covariance in Lecture 1.) & % Stat 504, Lecture 4 9 ' $ More generally, with k possible outcomes, the mean of Yi is = (1 , 2 , . . . , k ), and the covariance matrix is 2 3 1 (1 - 1 ) -1 2 -1 k 6 7 6 - 7 2 (1 - 2 ) -2 k 1 2 6 7 6 7. . . . 6 7 .. . . . 6 7 . . . . 4 5 -1 k -2 k k (1 - k ) & % Stat 504, Lecture 4 10 ' Using the fact that X = Y1 + Y2 + Yn , it immediately follows that the mean of X is n = (n1 , n2 , . . . , nk ) and the covariance matrix for X is 2 n1 (1 - 1 ) -n1 2 6 6 -n n2 (1 - 2 ) 1 2 6 6 . . 6 .. . . 6 . . . 4 -n1 k -n2 k 3 7 7 7 7. 7 7 5 $ -n1 k -n2 k . . . nk (1 - k ) Because the elements of X are constrained to sum to n, this covariance matrix is singular. If all the j 's are positive, then the covariance matrix has rank k - 1. Intuitively, this makes sense; the last element Xk can be replaced by n - X1 - X2 - - Xk-1 ; there are really only k - 1 "free" elements in X. If some elements of are zero, the rank drops by one for every zero element. & % Stat 504, Lecture 4 11 ' Parameter space. If we don't impose any restrictions on the parameter = (1 , 2 , . . . , k ) other than the necessary logically constraints j [0, 1], and 1 + 2 + k = 1, (2) then the parameter space is the set of all -vectors that satisfy (1) and (2). This set is called a simplex. In the special case of k = 3, we can visualize = (1 , 2 , 3 ) as a point in three-dimensional space. The simplex S is the triangular portion of a plane with vertices at (1, 0, 0), (0, 1, 0) and (0, 0, 1): j = 1, . . . , k (1) $ p3 (0,0,1) (0,1,0) p2 p 1 (1,0,0) & % Stat 504, Lecture 4 12 ' More generally, the simplex is a portion of a (k - 1)-dimensional hyperplane in k-dimensional space. Alternatively, we can replace k by 1 - 1 - 2 - - k-1 because it's not really a free parameter and view the simplex in (k - 1)-dimensional space. For example, with k = 3, we can replace 3 by 1 - 1 - 2 and view the parameter space as a triangle: $ p2 (0,1) (0,0) (1,0) p1 ML estimation. If X = Mult(n, ) and we observe X = x, then the loglikelihood function for is l(; x) = x1 log 1 + x2 log 2 + + xk log k . & % Stat 504, Lecture 4 13 ' $ Using multivariate calculus, it's easy to maximize this function subject to the constraint 1 + 2 + + k = 1; the maximum is achieved at p = = n-1 x (x1 /n, x2 /n, . . . , xk /n), the vector of sample proportions. The ML estimate for any individual j is pj = xj /n, and an approximate 95% confidence interval for j is r pj (1 - pj ) pj 1.96 n because Xj Bin(n, j ). & % Stat 504, Lecture 4 14 ' $ Fusing cells. We can collapse a multinomial vector by fusing cells (i.e. by adding some of the cell counts Xj together). If X = (X1 , X2 , . . . , Xk ) Mult(n, ) where = (1 , 2 , . . . , k ), then X = (X1 + X2 , X3 , X4 , . . . , Xk ) is also multinomial with the same index n and modified parameter = (1 + 2 , 3 , 4 , . . . , k ). In the multinomial experiment, we are simply fusing the events E1 and E2 into the single event "E1 or E2 ." Because these events are mutually exclusive, P (E1 or E2 ) = P (E1 ) +, P (E2 ) = 1 + 2 . & % Stat 504, Lecture 4 15 ' Partitioning the multinomial. We can also partion the multinomial by conditioning on (treating as fixed) the totals of subsets of cells. For example, consider the conditional distribution of X given that X1 + X2 X3 + X4 + + Xk = = z, n - z. $ The subvectors (X1 , X2 ) and (X3 , X4 , . . . , Xk ) are conditionally independent and multinomial, (X1 , X2 ) Mult z, h " (X3 , . . . , Xk ) Mult n - z, h " 1 , 2 1 +2 1 +2 "i , k 3 , . . . , + + + +k 3 3 k "i . The joint distribution of two or more independent multinomials is called "product-multinomial." If we condition on the sums of non-overlapping groups of cells of a multinomial vector, it's distribution splits into product multinomial. The parameter for each part of the product multinomial is a portion of the original vector, normalized to sum to one. & % Stat 504, Lecture 4 16 ' $ Relationship between the multinomial and Poisson. Suppose that X1 , X2 , . . . , Xk are independent Poisson random variables, X1 X2 . . . Xk P (k ), P (1 ), P (2 ), where the j 's are not necessarily equal. Then the conditional distribution of the vector X = (X1 , X2 , . . . , Xk ) given the total n = X1 + X2 + + Xk & % Stat 504, Lecture 4 17 ' is Mult(n, ), where = (1 , 2 , . . . , k ) j . 1 + 2 + + k That is, is simply the vector of j 's normalized to sum to one. j = This fact is important, because it implies that the unconditional distribution of (X1 , . . . , Xk ) can be factored into the product of two distributions: a Poisson distribution for the overall total, n P (1 + 2 + + k ), and a multinomial distribution for X = (X1 , X2 , . . . , Xk ) given n, X Mult(n, ). The likelihood factors into two independent functions, Pk one for j=1 j and the other for . The total n carries no information about and vice-versa. and $ & % Stat 504, Lecture 4 18 ' Therefore, likelihood-based inferences about are the same whether we regard X1 , . . . , Xk as sampled from k independent Poissons or from a single multinomial. That is, any estimates, tests, etc. for or functions of will be the same whether we regard n as random or fixed. Example. Suppose that you wait at a busy intersection for one hour and record the color of each vehicle as it drives by. Let X1 X2 X3 X4 X5 X6 X7 = = = = = = = number of white vehicles number of black vehicles number of silver vehicles number of red vehicles number of blue vehicles number of green vehicles number of any other color $ In this experiment, the total number of vehicles observed, n = X1 + X2 + + X7 , & % Stat 504, Lecture 4 19 ' $ is random. (It would have been fixed if, for example, we had decided to classify the first n = 500 vehicles we see. But because we decided to wait for one hour, the n is random.) In this case, it's reasonable to regard the Xj 's as independent Poisson random variables with means 1 , 2 , . . . , 7 . But if our interest lies not in the j 's but in the proportions of various colors in the vehicle population, inferences about these proportions will be the same whether we regard the sample size n as random or fixed. That is, we can proceed as if X = (X1 , . . . , X7 ) Mult(n, ) where = (1 , . . . , 7 ), even though n is actually random. Next time: Goodness of fit testing for one-way tables & %

Find millions of documents here - Study Guides, Homework Solutions, Papers, Exam Answer Keys and more. Course Hero has millions of course related materials that will enable you to learn better, faster and get an A in all your courses.
Below is a small sample set of documents: