# hw1sol - 10-708 Probabilistic Graphical Models Homework 1...

This preview shows pages 1–3. Sign up to view the full content.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: 10-708 Probabilistic Graphical Models Homework 1 Solutions Adapted from solutions of Anton Chechetka 1 1.1 It is known that P | = ( α ⊥ β | γ ) ⇐⇒ P ( α ∩ β | γ ) = P ( α | γ ) P ( β | γ ) (1) Now, if P ( α,β,γ ) = f ( α,γ ) g ( β,γ ) (2) then P ( α | γ ) = ∑ β ( f ( α,γ ) g ( β,γ )) ∑ α,β ( f ( α,γ ) g ( β,γ )) , P ( β | γ ) = ∑ α ( f ( α,γ ) g ( β,γ )) ∑ α,β ( f ( α,γ ) g ( β,γ )) (3) which after factoring out the constant terms in the summations turns to P ( α | γ ) = f ( α,γ ) ∑ β g ( β,γ ) ∑ α f ( α,γ ) ∑ β g ( β,γ ) = g ( α,γ ) ∑ α g ( α,γ ) , P ( β | γ ) = g ( β,γ ) ∑ α f ( α,γ ) ∑ β g ( β,γ ) ∑ α f ( α,γ ) = f ( β,γ ) ∑ β f ( β,γ ) (4) therefore P ( α | γ ) P ( β | γ ) = f ( α,γ ) g ( β,γ ) P β g ( β,γ ) P α f ( α,γ ) = P ( α,β,γ ) P β P α ( g ( β,γ ) f ( α,γ )) = P ( α,β,γ ) P β P α P ( α,β,γ ) = P ( α ∩ β | γ ) (5) and the right part of (1) holds, so indeed P | = ( α ⊥ β | γ ). Now let us prove the ”only if” part of the statement. Suppose P ( α ∩ β | γ ) = P ( α | γ ) P ( β | γ ) (6) then P ( α,β,γ ) = ( P ( α | γ ))( P ( β | γ ) P ( γ )) = f ( α,γ ) g ( β,γ ) (7) qed. 1.2 Note that the question asked whether f and g can be probability distributions over their sets of variables, and not whether they can be represented as marginals . Clearly, the latter implies the former, but the converse is not true. Also, note that p ( y | z ) is not a probability distribution over Y and Z . Proof by contradiction: Suppose f ( X,Z ) and g ( Y,Z ) are probability distributions. P ( x,y,z ) = f ( x,z ) g ( y,z ) P ( z ) = summationdisplay x ( f ( x,z )) summationdisplay y ( g ( y,z )) = f prime ( z ) g prime ( z ) summationdisplay z f prime ( z ) g prime ( z ) = 1 summationdisplay z f prime ( z ) = summationdisplay z g prime ( z ) = 1 1 We can assume that either f or g do not have all their masses at some z (if both have a point mass at some z = z , then P ( z ) = 1, which is a specific distribution. If both have point masses at different points, then P(Z) would not even be a distribution). Thus, we can assume that f prime ( z ) < 1 , ∀ z summationdisplay z f prime 2 ( z ) < summationdisplay z f prime ( z ) = 1 By Cauchy-Schwartz’s inequality, 1 = ( summationdisplay z f prime ( z ) g prime ( z )) 2 ≤ ( summationdisplay z f prime 2 ( z ))( summationdisplay z g prime 2 ( z )) < 1 which is a contradiction. Thus, it is not possible for both f and g to be probability distributions over their respective sets of variables. 1.3 1. Yes: P ( X,Y | Z ) = ∑ W P ( X,Y,W | Z ) = { use ( X ⊥ Y,W | Z ) } = ∑ W ( P ( X | Z ) P ( Y,W | Z )) = P ( X | Z ) ∑ W P ( Y,W | Z ) = P ( X | Z ) P ( Y | Z ) (8) 2. Yes, in fact ( X,Y ⊥ W | Z ) → ( X ⊥ W | Z ), as was shown in previous section, so ( X ⊥ Y | Z ) is redundant....
View Full Document

{[ snackBarMessage ]}

### Page1 / 9

hw1sol - 10-708 Probabilistic Graphical Models Homework 1...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online