300Chw3solutions

300Chw3solutions - Problem 1 Consider a natural exponential...

This preview shows pages 1–5. Sign up to view the full content.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Problem 1 / Consider a natural exponential family on (Xk, Bxk), with fe(1‘) = Z“1(9)h(\$)exp(9 4190)), Where ‘1 — :1; ex - at 00. 2(niLM>poa»< Let 7r(d0) be any prior such that the law of Z _1(9) is Cauchy. For example, if h(a:) = 1, t(:r) = x and k = 1, then f9(\$) is just the exponential density and Z‘1(0) = 9 has a Cauchy prior. This minimal case would be a natural model when the rate of an exponential random variable could be interpreted as a ratio of independent normals. In general, for any a; with t(x) = 0, the integral /tanwe=mm/z*mmwi 9 9 where the second factor is the ﬁrst moment of the Cauchy distribution, which is 00. as}; WﬂuéJ Ly Mam GMZIW (2:50” 2. Book problem 9, 36. Chapter 2, page 138 and 142. 9 Problem 9. Consider the experiment described in Example 2.54 on page 102. Let N be the number of observed H, X = U:=2{0,1}m, and X = (Z,Y1, . .. YN). Z/‘ (a) 52:3!er density of X given 9 = 6 with respect to counting measure on Our conditional density is given by 1 n_ 1 P((Z, Y1, . . . ,YN)|e = 0) = 59M(1 — 6) Mllz=1llN=n + §9k(1 — 0)N‘kllz=0.]lM=k]lyN=1 L/Uo) Let M be the number of observed successes among the Y,, that is, M = N 21:13/1” Show that (N,M) is suﬁicient. In particular, we do not need to keep track of Z. - Looking at the density above, we can rewrite it as 1 P Z,Y,...Y "'= =—M _ nhM 1 — (( 1 , N>|O 6) 2‘9 (1 0) ﬂz=1ﬂN=n + §6k(1— ‘9)N k112=011M=k11YN=1 _ 1 M N—M 1 - _ 56 (1— 0) HZ=111N=n + 59M“ — 6)N—M1Z=01M=leN=l _ M _ 1 _ (6 (1 — 0)N M) . 5 (112:111N=,, + lz=olM=k11yN=1) If we deﬁne l h(X) = 5 (112:111N=n + 11z=ollM=k11YN=1),g(9(N(X),M(X)) = 0M(1—6’)N"M 1 we see that we can write the conditional density as PWWFJWZKPWKM9=ﬂ=MMQ1NMDWW) so by the factorization theorem, M, N are sufﬁcient for 6, since the distribu— tions depends on 6 through a factor that depends on the data only through M, N. (c) Find the conditional distribution of Z given (N, M, 9), and show that it does / not depend on G P(Z,M,N) Recall that P(ZlM, N) = W. For the cases N = n,M 75 k, we must have Z: 1, so P(Z= llMaék,N=n) = 1. When Naén,M=ls, we must have Z = 0, so P(Z = O|M = k,N 75 n) = 1. For the case N = n,M = k, we need to think very slightly harder. Note that P(Z=1,M= k,N=n) = l<n)gk(.1 _6)n—k 2 k P(Z=0MzkN=n)=3(n_1>6k(1—6)n-k ’ ’ 2 k—l PMJ=kZV=nJ=1(n)Wﬂn—mm*+l<n—J>0H1—6W4 ’ 2 k 2 Is— So P(Z = le = k,N = n) = (3595) z = 1. Putting this all together, we have n (221—1 3:1,Th=~k,ﬁ=n k%i_tjl k—l ~ ,.. 11 11—1 Z=O,m=k)n=n P(Z = le = m, N = a) = (kWh—1) ~ ~ . l 1 Z=Lm#hn=n 1 z=0,ﬁ1=k’ﬁ,_7én 0 otherwise L/o Problem 36. In problem 9 on page 138, show that the family of distributions for observed data is an exponential family but that the suﬁicient statistic is not complete. How do you reconcile this with Theorem 2.74? Recall our density 1 P((Z,Y1, . - . ,YN)|@ = ‘9) = (6M(1 ‘ 6)N_M)'§(]1Z=11N=n + 11z=ollM=1cllvN=1) Mlog9+(N—M)10g(1—9) . l :e 2 (12:111N=n + 1lz=()]1M=klli/N=1) Mlog 1%9+Nlog(1—9) ' 3'; =6 2 (12:1 llN=n + 11z=011M=Ic 11YN=1) Note that this is an exponential family, with sufﬁcient statistics N ,M and pa— rameters log ﬁ, log(1 — 9). Next, we want to see that the sufﬁcient statistic is not complete. If N, M were complete, then since Z is ancillary to N, M as we showed above, we would have Z independent of N,M by Basu’s theorem. However, we know M,N are not independent of Z (for example, if Z = 0 then M = 1:, but if Z = 1 then M can take on other values), so thus N, M cannot be complete. How is this consistent with Theorem 2.74? One of the conditions of Theorem 2.74 for an exponential family to have complete sufﬁcient statistics is that the natural parameter space have an open rectangle inside. Our exponential family above is curved, since the “two” parameters log ﬁ and log(1 — 6) are both functions of 6, so the natural parameter space is just a one—dimensional curve Within the two—dimensional space, so it has no interior, and thus the interior cannot contain an open rectangle, so Theorem 2.74 does not apply (so there is no contradiction). 7H3 §o(ui§0b\ W'H" Maritov Chaim waki iv 53’9“) gawllado Problem 3 k/ Suppose T is classically sufﬁcient, and let 7rg(d0) be a prior measure on (9, 39). We claim that (6, T, X) is a Markov chain, which is to say, X and 9 are conditionally independent given T. For any B 6 Egg and any C’ 6 Be, the regular conditional probability can be expressed as P(X e B, a e CIT = t.) = / P(X e B|0 = 19,1“ = t)7rng=t(d19) C' = /0 PM 6 BIT = WWW) But by classical sufﬁciency, the integrand is a function r(B, t) with the properties of a regular conditional probability, which does not depend on 19, therefore, we can take it out of the integral to obtain, P(X 6 3,0 6 CIT = t) = r(B,t) /7rng=t(d19) C = P(X e BIT = t)P(6 e C|T = t). In the second identity, we used the fact that r(B,t) is also the regular conditional probability of X given T, after marginalizing over 0. This proves our claim. Now, we can use the general fact that if (6,X , T) is a Markov chain, then so is (X, T, 6). This implies that the posterior of 6 given X and T is the same as the posterior of 0 given just T. Since T is a function of X, then for any B E 89, we have W9|X1T(B|x, T(:1:)) = 7r9|X(le). Putting this together, we conclude that for some version of the posteriors, almost surely 7r9|T(BIT(a:)) = «ng(B|x), which is equivalent to Bayes sufﬁciency. fatu'gah ‘cﬁt(0‘4-’;"9 m (VDVL pé Z!‘% in Scum-ELI Wank! 4'9 KOCWI‘J' Mukl’lerjéé 3. / The proof is, based on the lines of theorem 2.14 of page 86 of the book Theory of Statistics by Mark Schervish. Let B E Bx. Let 7-} and 7-19 be the a—algebra generated by T and {T , 0} respectively. By the law of total probability, (or by the tower property of measure theory, we have E(H(X E Blft) = E[E{H(X E B)|-7:t,0}’-7:t] By existence of regular conditional probability, this is same as #mﬂMT=w=[fan=aaewmw where New is the conditional distribution of 0 given T. Noting that T is classically sufﬁcient, we have that P9(B|T) (14-5 T(B, t), where r(., t) is the regular conditional probability of X given T = t. Plugging in T(B, t) for Pg(B[T = t) in the above relation, we have lqMMT=n=Awaowwwm=maa Thus we have MX|T(B|T = t) = 7(3) t) = P9(BIT = 75) Now the LHS is the conditional distribution of X given T = t, and the RHS is the conditional distribution of X given T, 0. Since the two are same, it follows that the conditional distribution of X given T, 6 is same as the conditional distribution of X given T, and so by theorem B.64 we have X and 9 are conditionally independent given T. By theorem B.61, this implies that for any set D e 9, we have P(@ E DIX, T) “—3 P(@ e DlT) Finally nothing that T = T(X) is a measurable function of X, we have by corollary B.74 that P(@ E D|X,T) 9:8 P(9 E D|X) and so P(@ e D|X) “=3 P(@ e DlT) This is the deﬁnition of Bayesina sufﬁciency. and so T is sufﬁcient in the Bayesian sense. ...
View Full Document

{[ snackBarMessage ]}

What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern