This preview shows pages 1–3. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: 10708 Probabilistic Graphical Models Homework 1 Solutions Adapted from solutions of Anton Chechetka 1 1.1 It is known that P  = ( α ⊥ β  γ ) ⇐⇒ P ( α ∩ β  γ ) = P ( α  γ ) P ( β  γ ) (1) Now, if P ( α,β,γ ) = f ( α,γ ) g ( β,γ ) (2) then P ( α  γ ) = ∑ β ( f ( α,γ ) g ( β,γ )) ∑ α,β ( f ( α,γ ) g ( β,γ )) , P ( β  γ ) = ∑ α ( f ( α,γ ) g ( β,γ )) ∑ α,β ( f ( α,γ ) g ( β,γ )) (3) which after factoring out the constant terms in the summations turns to P ( α  γ ) = f ( α,γ ) ∑ β g ( β,γ ) ∑ α f ( α,γ ) ∑ β g ( β,γ ) = g ( α,γ ) ∑ α g ( α,γ ) , P ( β  γ ) = g ( β,γ ) ∑ α f ( α,γ ) ∑ β g ( β,γ ) ∑ α f ( α,γ ) = f ( β,γ ) ∑ β f ( β,γ ) (4) therefore P ( α  γ ) P ( β  γ ) = f ( α,γ ) g ( β,γ ) P β g ( β,γ ) P α f ( α,γ ) = P ( α,β,γ ) P β P α ( g ( β,γ ) f ( α,γ )) = P ( α,β,γ ) P β P α P ( α,β,γ ) = P ( α ∩ β  γ ) (5) and the right part of (1) holds, so indeed P  = ( α ⊥ β  γ ). Now let us prove the ”only if” part of the statement. Suppose P ( α ∩ β  γ ) = P ( α  γ ) P ( β  γ ) (6) then P ( α,β,γ ) = ( P ( α  γ ))( P ( β  γ ) P ( γ )) = f ( α,γ ) g ( β,γ ) (7) qed. 1.2 Note that the question asked whether f and g can be probability distributions over their sets of variables, and not whether they can be represented as marginals . Clearly, the latter implies the former, but the converse is not true. Also, note that p ( y  z ) is not a probability distribution over Y and Z . Proof by contradiction: Suppose f ( X,Z ) and g ( Y,Z ) are probability distributions. P ( x,y,z ) = f ( x,z ) g ( y,z ) P ( z ) = summationdisplay x ( f ( x,z )) summationdisplay y ( g ( y,z )) = f prime ( z ) g prime ( z ) summationdisplay z f prime ( z ) g prime ( z ) = 1 summationdisplay z f prime ( z ) = summationdisplay z g prime ( z ) = 1 1 We can assume that either f or g do not have all their masses at some z (if both have a point mass at some z = z , then P ( z ) = 1, which is a specific distribution. If both have point masses at different points, then P(Z) would not even be a distribution). Thus, we can assume that f prime ( z ) < 1 , ∀ z summationdisplay z f prime 2 ( z ) < summationdisplay z f prime ( z ) = 1 By CauchySchwartz’s inequality, 1 = ( summationdisplay z f prime ( z ) g prime ( z )) 2 ≤ ( summationdisplay z f prime 2 ( z ))( summationdisplay z g prime 2 ( z )) < 1 which is a contradiction. Thus, it is not possible for both f and g to be probability distributions over their respective sets of variables. 1.3 1. Yes: P ( X,Y  Z ) = ∑ W P ( X,Y,W  Z ) = { use ( X ⊥ Y,W  Z ) } = ∑ W ( P ( X  Z ) P ( Y,W  Z )) = P ( X  Z ) ∑ W P ( Y,W  Z ) = P ( X  Z ) P ( Y  Z ) (8) 2. Yes, in fact ( X,Y ⊥ W  Z ) → ( X ⊥ W  Z ), as was shown in previous section, so ( X ⊥ Y  Z ) is redundant....
View
Full
Document
This note was uploaded on 05/25/2008 for the course MACHINE LE 10708 taught by Professor Carlosgustin during the Fall '07 term at Carnegie Mellon.
 Fall '07
 CarlosGustin

Click to edit the document details