This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: 10708 Probabilistic Graphical Models Homework 1 Solutions Adapted from solutions of Anton Chechetka 1 1.1 It is known that P  = ( α ⊥ β  γ ) ⇐⇒ P ( α ∩ β  γ ) = P ( α  γ ) P ( β  γ ) (1) Now, if P ( α,β,γ ) = f ( α,γ ) g ( β,γ ) (2) then P ( α  γ ) = ∑ β ( f ( α,γ ) g ( β,γ )) ∑ α,β ( f ( α,γ ) g ( β,γ )) , P ( β  γ ) = ∑ α ( f ( α,γ ) g ( β,γ )) ∑ α,β ( f ( α,γ ) g ( β,γ )) (3) which after factoring out the constant terms in the summations turns to P ( α  γ ) = f ( α,γ ) ∑ β g ( β,γ ) ∑ α f ( α,γ ) ∑ β g ( β,γ ) = g ( α,γ ) ∑ α g ( α,γ ) , P ( β  γ ) = g ( β,γ ) ∑ α f ( α,γ ) ∑ β g ( β,γ ) ∑ α f ( α,γ ) = f ( β,γ ) ∑ β f ( β,γ ) (4) therefore P ( α  γ ) P ( β  γ ) = f ( α,γ ) g ( β,γ ) P β g ( β,γ ) P α f ( α,γ ) = P ( α,β,γ ) P β P α ( g ( β,γ ) f ( α,γ )) = P ( α,β,γ ) P β P α P ( α,β,γ ) = P ( α ∩ β  γ ) (5) and the right part of (1) holds, so indeed P  = ( α ⊥ β  γ ). Now let us prove the ”only if” part of the statement. Suppose P ( α ∩ β  γ ) = P ( α  γ ) P ( β  γ ) (6) then P ( α,β,γ ) = ( P ( α  γ ))( P ( β  γ ) P ( γ )) = f ( α,γ ) g ( β,γ ) (7) qed. 1.2 Note that the question asked whether f and g can be probability distributions over their sets of variables, and not whether they can be represented as marginals . Clearly, the latter implies the former, but the converse is not true. Also, note that p ( y  z ) is not a probability distribution over Y and Z . Proof by contradiction: Suppose f ( X,Z ) and g ( Y,Z ) are probability distributions. P ( x,y,z ) = f ( x,z ) g ( y,z ) P ( z ) = summationdisplay x ( f ( x,z )) summationdisplay y ( g ( y,z )) = f prime ( z ) g prime ( z ) summationdisplay z f prime ( z ) g prime ( z ) = 1 summationdisplay z f prime ( z ) = summationdisplay z g prime ( z ) = 1 1 We can assume that either f or g do not have all their masses at some z (if both have a point mass at some z = z , then P ( z ) = 1, which is a specific distribution. If both have point masses at different points, then P(Z) would not even be a distribution). Thus, we can assume that f prime ( z ) < 1 , ∀ z summationdisplay z f prime 2 ( z ) < summationdisplay z f prime ( z ) = 1 By CauchySchwartz’s inequality, 1 = ( summationdisplay z f prime ( z ) g prime ( z )) 2 ≤ ( summationdisplay z f prime 2 ( z ))( summationdisplay z g prime 2 ( z )) < 1 which is a contradiction. Thus, it is not possible for both f and g to be probability distributions over their respective sets of variables. 1.3 1. Yes: P ( X,Y  Z ) = ∑ W P ( X,Y,W  Z ) = { use ( X ⊥ Y,W  Z ) } = ∑ W ( P ( X  Z ) P ( Y,W  Z )) = P ( X  Z ) ∑ W P ( Y,W  Z ) = P ( X  Z ) P ( Y  Z ) (8) 2. Yes, in fact ( X,Y ⊥ W  Z ) → ( X ⊥ W  Z ), as was shown in previous section, so ( X ⊥ Y  Z ) is redundant....
View
Full Document
 Fall '07
 CarlosGustin
 Conditional Probability

Click to edit the document details