Unformatted text preview: Massachusetts Institute of Technology 6.042J/18.062J, Fall ’02: Mathematics for Computer Science Prof. Albert Meyer and Dr. Radhika Nagpal Solutions to InClass Problems — Week 13, Wed Problem 1. Let R be the number of heads that come up when we toss n independent coins, where each coin comes up heads with probability p. The random variable R has a binomial distribution. Using the formula for variance of sums, show that Var [R] = np(1 − p). Solution. From the course notes: Recall that if a random variable, R, has a binomial distribution, then �� nk Pr {R = k } = p (1 − p)n−k k where n and p are parameters such that n ≥ 1 and 0 < p < 1. We can think of R as the sum of n independent Bernoulli variables. Formally, we can write R = R1 + R2 + · · · + Rn where � 1 with probability p, Ri = 0 with probability 1 − p. Now we can compute the variance of the binomially distributed variable R. Var [R] = Var [R1 ] + Var [R2 ] + . . . + Var [Rn ] = n Var [R1 ] � 2� = n(E R1 − E2 [R1 ]) = n(E [R1 ] − E [R1 ])
2 (Theorem 4.1) (Var [Ri ] = Var [Rj ]) (Def. of variance)
2 (R1 = R1 ) = n(p − p2 ) = np(1 − p). (E [R1 ] = Pr {R1 = 1} = p) � This shows that the binomial distribution has variance np(1−p) and standard deviation np(1 − p). In the special case of an unbiased binomial distribution (p = 1/2), the variance is n/4 and the stan √ � dard deviation is n/2. Copyright © 2002, Dr. Radhika Nagpal. 2 Solutions to InClass Problems — Week 13, Wed Problem 2. Provide an example that shows that the variance of the sum of two random variables is not necessarily equal to the sum of their variances, when the random variables are not indepen dent. Solution. A dramatic example is to take Y = −X . Then the sum of the two random variables is 0, so the variance of X + Y is 0. But the sum of the variances is 2 Var [X ], since Y has the same variance as X . Another example: let X be equal to 1 if a ﬂipped coin is heads, 0 otherwise. Similarly, let Y be 1 if the same coin comes up tails. Then Var [X ] = Var [Y ] = 1/4, but X + Y = 1, so Var [X + Y ] = 0. � Problem 3. The hatcheck staff has had a long day, and at the end of the party they decide to return people’s hats at random. Suppose that n people have their hats returned at random. We previously showed that the expected number of people who get their own hat back is 1, irrespective of the total number of people. In this problem we will calculate the variance in the number of people who get their hat back. � Let Xi = 1 if the ith person gets his or her own hat back and 0 otherwise. Let Sn = n Xi , so Sn i=1 is the total number of people who get their own hat back. Show that �� (a) E Xi2 = 1/n. Solution. Xi = �1 w� probability 1/n and 0 otherwise. Thus Xi2 = 1 with probability 1/n and 0 ith 2 = 1/n. otherwise. So E Xi � (b) E [Xi Xj ] = 1/n(n − 1) for i �= j . Solution. The probability that Xi and Xj are both 1 is 1/n · 1/(n − 1) = 1/n(n − 1). Thus Xi Xj = 1 with probability 1/n(n − 1), and is zero otherwise. So E [Xi Xj ] = 1/n(n − 1). � � 2� (c) E Sn = 2. Hint: Use (a) and (b). Solution. � � � �� � 2� E Sn = E Xi2 + E [Xi Xj ]
i i j �=i 1 1 = n · + n(n − 1) · n n(n − 1) = 2. � Solutions to InClass Problems — Week 13, Wed (d) Var [Sn ] = 1. Solution. � 2� Var [Sn ] = E Sn − E2 [Sn ] = 2 − (n(1/n))2 = 2−1 = 1. 3 � (e) Explain why you can not use the variance of sums formula to calculate Var [Sn ]. Solution. The indicator random variables, Xi , are not even pairwise independent. This can be seen by comparing the marginal and conditional probability of a particular person, Alice, getting her hat back. The marginal probability, unconditioned on any other events, is 1/n as we’ve com puted before. However, if compute this probability conditioned on the event that a second person, Bob, got his hat back, we ﬁnd that the probability of Alice getting her hat back is 1/(n − 1). � (f) Using Chebyshev’s Inequality, show that Pr {Sn ≥ 11} ≤ .01 for any n ≥ 11. Solution. Pr {Sn ≥ 11} = Pr {Sn − E [Sn ] ≥ 11 − E [Sn ]} = Pr {Sn − E [Sn ] ≥ 10} Var [Sn ] ≤ = .01 102 Note that the Xi ’s are Bernoulli variables but are not independent, so Sn does not have a binomial distribution and the binomial estimates from Lecture Notes do not apply. � Problem 4. Prove that Var [X + Y + Z ] = Var [X ] + Var [Y ] + Var [Z ], if X, Y, Z are pairwise inde pendent. Explain why pairwise independence is sufﬁcient. Reminder: If two random variable are independent, then E [Xi Xj ] = E [Xi ] E [Xj ]. Given a set of random variables, pairwise independence implies that every possible pair of random variables is independent. Solution. Let R = X + Y + Z . We can compute E2 [R] as follows: E2 [R] = (E [X + Y + Z ])2 = (E [X ] + E [Y ] + E [Z ])2 = E2 [X ] + E2 [Y ] + E2 [Z ] + 2 E [X ] E [Y ] + 2 E [X ] E [Z ] + 2 E [Y ] E [Z ] 4 Solutions to InClass Problems — Week 13, Wed �� Computing E R2 we get: � � � � E R2 = E (X + Y + Z )2 � � = E X 2 + Y 2 + Z 2 + 2XY + 2XZ + 2Y Z �� �� �� = E X 2 + E Y 2 + E Z 2 + 2 E [XY ] + 2 E [XZ ] + 2 E [Y Z ] �� �� �� = E X 2 + E Y 2 + E Z 2 + 2 E [X ] E [Y ] + 2 E [X ] E [Z ] + 2 E [Y ] E [Z ] Notice that the last step is only valid because the random variables are pairwise independent. Finally, we can compute Var [R]. We can begin immediately by cancelling out the cross terms leaving us with: �� Var [R] = E R2 − E2 [R] �� �� �� = E X 2 + E Y 2 + E Z 2 − (E2 [X ] + E2 [Y ] + E2 [Z ]) �� �� �� = E X 2 − E2 [X ] + E Y 2 − E2 [Y ] + E Z 2 − E2 [Z ] = Var [X ] + Var [Y ] + Var [Z ] , thus concluding the proof. � A Appendix Random variables R1 , R2 , . . . are mutually independent iff � � � � Pr [Ri = xi ] = Pr {Ri = xi } ,
i i for all x1 , x2 , · · · ∈ R. They are k wise independent iff {Ri  i ∈ J } are mutually independent for all subsets J ⊂ N with J  = k . Theorem (Expectation of a Product). If R1 , R2 , . . . , Rn are mutually independent, then E [R1 · R2 · · · · · Rn ] = E [R1 ] · E [R2 ] · · · E [Rn ] . The variance, Var [R], of a random variable, R, is: � � Var [R] ::= E (R − E [R])2 . Variance can also be equivalently deﬁned as: �� Var [R] ::= E R2 − E2 [R] , Lemma. For a, b ∈ R, Var [aR + b] = a2 Var [R] Solutions to InClass Problems — Week 13, Wed Theorem 4.1. If R1 , R2 , . . . , Rn are pairwise independent random variables, then Var [R1 + R2 + · · · + Rn ] = Var [R1 ] + Var [R2 ] + · · · + Var [Rn ] . Theorem (Markov’s Theorem). If R is a nonnegative random variable, then for all x > 0 Pr {R ≥ x} ≤ An alternative formulation is Pr {R ≥ x E [R]} ≤ 1 . x E [R] . x 5 Theorem (Chebyshev). Let R be a random variable, and let x be a positive real number. Then Pr {R − E [R] ≥ x} ≤ An alternative formulation is Pr {R − E [R] ≥ xσR } ≤ where σR ::= � Var [R] is the standard deviation of R. 1 , x2 Var [R] . x2 ...
View
Full
Document
This note was uploaded on 01/19/2010 for the course MATH 55 taught by Professor Strain during the Fall '08 term at Berkeley.
 Fall '08
 STRAIN
 Math, Variance

Click to edit the document details