hw1sol - 10-708 Probabilistic Graphical Models Homework 1...

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: 10-708 Probabilistic Graphical Models Homework 1 Solutions Adapted from solutions of Anton Chechetka 1 1.1 It is known that P | = ( α ⊥ β | γ ) ⇐⇒ P ( α ∩ β | γ ) = P ( α | γ ) P ( β | γ ) (1) Now, if P ( α,β,γ ) = f ( α,γ ) g ( β,γ ) (2) then P ( α | γ ) = ∑ β ( f ( α,γ ) g ( β,γ )) ∑ α,β ( f ( α,γ ) g ( β,γ )) , P ( β | γ ) = ∑ α ( f ( α,γ ) g ( β,γ )) ∑ α,β ( f ( α,γ ) g ( β,γ )) (3) which after factoring out the constant terms in the summations turns to P ( α | γ ) = f ( α,γ ) ∑ β g ( β,γ ) ∑ α f ( α,γ ) ∑ β g ( β,γ ) = g ( α,γ ) ∑ α g ( α,γ ) , P ( β | γ ) = g ( β,γ ) ∑ α f ( α,γ ) ∑ β g ( β,γ ) ∑ α f ( α,γ ) = f ( β,γ ) ∑ β f ( β,γ ) (4) therefore P ( α | γ ) P ( β | γ ) = f ( α,γ ) g ( β,γ ) P β g ( β,γ ) P α f ( α,γ ) = P ( α,β,γ ) P β P α ( g ( β,γ ) f ( α,γ )) = P ( α,β,γ ) P β P α P ( α,β,γ ) = P ( α ∩ β | γ ) (5) and the right part of (1) holds, so indeed P | = ( α ⊥ β | γ ). Now let us prove the ”only if” part of the statement. Suppose P ( α ∩ β | γ ) = P ( α | γ ) P ( β | γ ) (6) then P ( α,β,γ ) = ( P ( α | γ ))( P ( β | γ ) P ( γ )) = f ( α,γ ) g ( β,γ ) (7) qed. 1.2 Note that the question asked whether f and g can be probability distributions over their sets of variables, and not whether they can be represented as marginals . Clearly, the latter implies the former, but the converse is not true. Also, note that p ( y | z ) is not a probability distribution over Y and Z . Proof by contradiction: Suppose f ( X,Z ) and g ( Y,Z ) are probability distributions. P ( x,y,z ) = f ( x,z ) g ( y,z ) P ( z ) = summationdisplay x ( f ( x,z )) summationdisplay y ( g ( y,z )) = f prime ( z ) g prime ( z ) summationdisplay z f prime ( z ) g prime ( z ) = 1 summationdisplay z f prime ( z ) = summationdisplay z g prime ( z ) = 1 1 We can assume that either f or g do not have all their masses at some z (if both have a point mass at some z = z , then P ( z ) = 1, which is a specific distribution. If both have point masses at different points, then P(Z) would not even be a distribution). Thus, we can assume that f prime ( z ) < 1 , ∀ z summationdisplay z f prime 2 ( z ) < summationdisplay z f prime ( z ) = 1 By Cauchy-Schwartz’s inequality, 1 = ( summationdisplay z f prime ( z ) g prime ( z )) 2 ≤ ( summationdisplay z f prime 2 ( z ))( summationdisplay z g prime 2 ( z )) < 1 which is a contradiction. Thus, it is not possible for both f and g to be probability distributions over their respective sets of variables. 1.3 1. Yes: P ( X,Y | Z ) = ∑ W P ( X,Y,W | Z ) = { use ( X ⊥ Y,W | Z ) } = ∑ W ( P ( X | Z ) P ( Y,W | Z )) = P ( X | Z ) ∑ W P ( Y,W | Z ) = P ( X | Z ) P ( Y | Z ) (8) 2. Yes, in fact ( X,Y ⊥ W | Z ) → ( X ⊥ W | Z ), as was shown in previous section, so ( X ⊥ Y | Z ) is redundant....
View Full Document

This note was uploaded on 05/25/2008 for the course MACHINE LE 10708 taught by Professor Carlosgustin during the Fall '07 term at Carnegie Mellon.

Page1 / 9

hw1sol - 10-708 Probabilistic Graphical Models Homework 1...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online