This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Problem 1 / Consider a natural exponential family on (Xk, Bxk), with
fe(1‘) = Z“1(9)h($)exp(9 4190)), Where ‘1 — :1; ex  at 00.
2(niLM>poa»< Let 7r(d0) be any prior such that the law of Z _1(9) is Cauchy. For example,
if h(a:) = 1, t(:r) = x and k = 1, then f9($) is just the exponential density
and Z‘1(0) = 9 has a Cauchy prior. This minimal case would be a natural
model when the rate of an exponential random variable could be interpreted
as a ratio of independent normals. In general, for any a; with t(x) = 0, the
integral /tanwe=mm/z*mmwi
9 9 where the second factor is the ﬁrst moment of the Cauchy distribution, which
is 00. as}; WﬂuéJ Ly Mam GMZIW (2:50” 2. Book problem 9, 36. Chapter 2, page 138 and 142. 9 Problem 9. Consider the experiment described in Example 2.54 on page 102.
Let N be the number of observed H, X = U:=2{0,1}m, and X = (Z,Y1, . .. YN). Z/‘ (a) 52:3!er density of X given 9 = 6 with respect to counting measure on Our conditional density is given by 1 n_ 1
P((Z, Y1, . . . ,YN)e = 0) = 59M(1 — 6) Mllz=1llN=n + §9k(1 — 0)N‘kllz=0.]lM=k]lyN=1 L/Uo) Let M be the number of observed successes among the Y,, that is, M = N
21:13/1” Show that (N,M) is suﬁicient. In particular, we do not need to
keep track of Z.  Looking at the density above, we can rewrite it as
1
P Z,Y,...Y "'= =—M _ nhM 1 —
(( 1 , N>O 6) 2‘9 (1 0) ﬂz=1ﬂN=n + §6k(1— ‘9)N k112=011M=k11YN=1 _ 1 M N—M 1 
_ 56 (1— 0) HZ=111N=n + 59M“ — 6)N—M1Z=01M=leN=l _ M _ 1
_ (6 (1 — 0)N M) . 5 (112:111N=,, + lz=olM=k11yN=1) If we deﬁne l
h(X) = 5 (112:111N=n + 11z=ollM=k11YN=1),g(9(N(X),M(X)) = 0M(1—6’)N"M 1 we see that we can write the conditional density as
PWWFJWZKPWKM9=ﬂ=MMQ1NMDWW) so by the factorization theorem, M, N are sufﬁcient for 6, since the distribu—
tions depends on 6 through a factor that depends on the data only through M, N.
(c) Find the conditional distribution of Z given (N, M, 9), and show that it does
/ not depend on G
P(Z,M,N) Recall that P(ZlM, N) = W. For the cases N = n,M 75 k, we must
have Z: 1, so P(Z= llMaék,N=n) = 1. When Naén,M=ls, we must
have Z = 0, so P(Z = OM = k,N 75 n) = 1. For the case N = n,M = k,
we need to think very slightly harder. Note that P(Z=1,M= k,N=n) = l<n)gk(.1 _6)n—k 2 k
P(Z=0MzkN=n)=3(n_1>6k(1—6)nk
’ ’ 2 k—l
PMJ=kZV=nJ=1(n)Wﬂn—mm*+l<n—J>0H1—6W4
’ 2 k 2 Is— So P(Z = le = k,N = n) = (3595) z = 1. Putting this all together, we have n (221—1 3:1,Th=~k,ﬁ=n
k%i_tjl
k—l ~ ,..
11 11—1 Z=O,m=k)n=n
P(Z = le = m, N = a) = (kWh—1) ~ ~ . l 1 Z=Lm#hn=n 1 z=0,ﬁ1=k’ﬁ,_7én 0 otherwise L/o Problem 36. In problem 9 on page 138, show that the family of distributions
for observed data is an exponential family but that the suﬁicient statistic is not
complete. How do you reconcile this with Theorem 2.74? Recall our density 1
P((Z,Y1, .  . ,YN)@ = ‘9) = (6M(1 ‘ 6)N_M)'§(]1Z=11N=n + 11z=ollM=1cllvN=1) Mlog9+(N—M)10g(1—9) . l :e 2 (12:111N=n + 1lz=()]1M=klli/N=1) Mlog 1%9+Nlog(1—9) ' 3'; =6 2 (12:1 llN=n + 11z=011M=Ic 11YN=1) Note that this is an exponential family, with sufﬁcient statistics N ,M and pa—
rameters log ﬁ, log(1 — 9). Next, we want to see that the sufﬁcient statistic is not complete. If N, M were
complete, then since Z is ancillary to N, M as we showed above, we would have
Z independent of N,M by Basu’s theorem. However, we know M,N are not
independent of Z (for example, if Z = 0 then M = 1:, but if Z = 1 then M can
take on other values), so thus N, M cannot be complete. How is this consistent with Theorem 2.74? One of the conditions of Theorem 2.74
for an exponential family to have complete sufﬁcient statistics is that the natural
parameter space have an open rectangle inside. Our exponential family above
is curved, since the “two” parameters log ﬁ and log(1 — 6) are both functions
of 6, so the natural parameter space is just a one—dimensional curve Within the
two—dimensional space, so it has no interior, and thus the interior cannot contain
an open rectangle, so Theorem 2.74 does not apply (so there is no contradiction). 7H3 §o(ui§0b\ W'H" Maritov Chaim
waki iv 53’9“) gawllado Problem 3 k/ Suppose T is classically sufﬁcient, and let 7rg(d0) be a prior measure on
(9, 39). We claim that (6, T, X) is a Markov chain, which is to say, X and
9 are conditionally independent given T. For any B 6 Egg and any C’ 6 Be,
the regular conditional probability can be expressed as P(X e B, a e CIT = t.) = / P(X e B0 = 19,1“ = t)7rng=t(d19)
C'
= /0 PM 6 BIT = WWW) But by classical sufﬁciency, the integrand is a function r(B, t) with the
properties of a regular conditional probability, which does not depend on 19,
therefore, we can take it out of the integral to obtain, P(X 6 3,0 6 CIT = t) = r(B,t) /7rng=t(d19)
C
= P(X e BIT = t)P(6 e CT = t). In the second identity, we used the fact that r(B,t) is also the regular
conditional probability of X given T, after marginalizing over 0. This proves
our claim. Now, we can use the general fact that if (6,X , T) is a Markov
chain, then so is (X, T, 6). This implies that the posterior of 6 given X and
T is the same as the posterior of 0 given just T. Since T is a function of X,
then for any B E 89, we have W9X1T(Bx, T(:1:)) = 7r9X(le). Putting this
together, we conclude that for some version of the posteriors, almost surely
7r9T(BIT(a:)) = «ng(Bx), which is equivalent to Bayes sufﬁciency. fatu'gah ‘cﬁt(0‘4’;"9 m (VDVL pé Z!‘% in ScumELI
Wank! 4'9 KOCWI‘J' Mukl’lerjéé 3.
/ The proof is, based on the lines of theorem 2.14 of page 86 of the book Theory of Statistics
by Mark Schervish. Let B E Bx. Let 7} and 719 be the a—algebra generated by T and {T , 0} respectively. By
the law of total probability, (or by the tower property of measure theory, we have E(H(X E Blft) = E[E{H(X E B)7:t,0}’7:t] By existence of regular conditional probability, this is same as #mﬂMT=w=[fan=aaewmw where New is the conditional distribution of 0 given T. Noting that T is classically sufﬁcient, we have that P9(BT) (145 T(B, t), where r(., t) is the
regular conditional probability of X given T = t. Plugging in T(B, t) for Pg(B[T = t) in the
above relation, we have lqMMT=n=Awaowwwm=maa Thus we have
MXT(BT = t) = 7(3) t) = P9(BIT = 75) Now the LHS is the conditional distribution of X given T = t, and the RHS is the conditional
distribution of X given T, 0. Since the two are same, it follows that the conditional distribution
of X given T, 6 is same as the conditional distribution of X given T, and so by theorem B.64
we have X and 9 are conditionally independent given T. By theorem B.61, this implies that for any set D e 9, we have
P(@ E DIX, T) “—3 P(@ e DlT) Finally nothing that T = T(X) is a measurable function of X, we have by corollary B.74 that
P(@ E DX,T) 9:8 P(9 E DX) and so
P(@ e DX) “=3 P(@ e DlT) This is the deﬁnition of Bayesina sufﬁciency. and so T is sufﬁcient in the Bayesian sense. ...
View
Full Document
 Spring '09

Click to edit the document details