This preview shows pages 1–3. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Probablistic Graphical Models, Spring 2007 Homework 1 solutions 1 Representation Solution due to Steve Gardiner Consider N +1 binary random variables X 1 ,...X N ,Y that have the following conditional independencies: i,jX i X j  Y 1. Consider the following conjecture: Conjecture 1. i,j,k X i ,X j X k  Y The conjecture is false; a counterexample follows. Consider a model where N = 3 and X 1 ,X 2 and Y are all i.i.d. uniform Bernoulli random variables, and X 3 is deterministically generated as X 3 = Y ( X 1 X 2 ). (where refers to the logical XOR operator) The pairwise conditional independencies hold, namely, X 1 X 2  Y,X 1 X 3  Y and X 2 X 3  Y , however it is not the case that X 1 ,X 2 X 3  Y 2. Suppose you wish to store the joint probability distribution of these N + 1 variables as a single table. How many parameters will you need? 2 N +1 1 3. An undirected graphical model that encodes the required conditional independencies is Y X 1 X 2 X 3 .... X N 2 Gaussians Solution due to Steve Gardiner 1. Show that for any joint density function P ( X,Y ), if X Y then Cov ( X,Y ) = 0. Lemma 2. If X Y then E [ XY ] = [ EX ][ EY ] Proof. I show the result for continuous random variables X and Y ; the proof for discrete random variables uses summation symbols instead of integration but in the same manner. 1 E [ XY ] = integraldisplay integraldisplay xyP ( x,y ) dxdy = integraldisplay integraldisplay xyP ( x ) P ( y ) dxdy (by independence) = integraldisplay yP ( y ) integraldisplay xP ( x ) dxdy = bracketleftBig integraldisplay yP ( y ) dy bracketrightBigbracketleftBig integraldisplay xP ( x ) dx bracketrightBig = [ EX ][ EY ] Corollary 3. If X Y then Cov ( X,Y ) = 0 Proof. Cov ( X,Y ) = E bracketleftbig ( X EX )( Y EY ) bracketrightbig = E bracketleftbig XY Y EX XEY + [ EX ][ EY ] bracketrightbig = E [ XY ] E [ Y EX ] E [ XEY ] + E bracketleftbig [ EX ][ EY ] bracketrightbig = E [ XY ] [ EY ][ EX ] [ EX ][ EY ] + [ EX ][ EY ] = E [ XY ] [ EX ][ EY ] = E [ XY ] E [ XY ] (by Lemma 2) = 0 2. To Show: Suppose P ( X,Y ) a Gaussian distribution. Then Cov ( X,Y ) = 0 = X Y Proof. The joint distribution of (X,Y) is given by P ( x,y ) = 1 2 X Y radicalbig 1 2 exp bracketleftBig 1 2(1 2 ) bracketleftbig 2 ( x EX )( y EY ) X Y ( x EX ) 2 2 X ( y EY ) 2 2 Y bracketrightbig bracketrightBig...
View Full
Document
 Fall '07
 CarlosGustin

Click to edit the document details