*This preview shows
pages
1–8. Sign up
to
view the full content.*

This
** preview**
has intentionally

**sections.**

*blurred***to view the full version.**

*Sign up*This
** preview**
has intentionally

**sections.**

*blurred***to view the full version.**

*Sign up*This
** preview**
has intentionally

**sections.**

*blurred***to view the full version.**

*Sign up*This
** preview**
has intentionally

**sections.**

*blurred***to view the full version.**

*Sign up*
**Unformatted text preview: **Machine Learning Srihari 1 Semantics of Markov Networks Sargur Srihari [email protected] Machine Learning Srihari Topics • Independencies in Markov Networks – Global and Local Independencies – From Distributions to Graphs • Parameterization – Factor Graphs – Log-linear Models – Features (Ising, Boltzmann) – Overparameterization & Canonical Parameterization • Bayesian Networks and Markov Networks – From BN to MN: Moralized graphs – From MN to BN: Chordal graphs 2 Machine Learning Srihari Global Independence Definition H is a Markov network X 1 ,..X k is a path in H Z : set of observed variables 1.Path is active given Z if none of the X i is in Z – If B and D are observed, path A,C is inactive 2.Set of nodes Z separates sets X and Y – If no active path between any node X ∈ X and Y ∈ Y 3.Global independencies associated with H are – I( H )={(X ⊥ Y|Z): sep H (X,Y|Z)} – Independencies in I (H) guaranteed to hold for every distribution P over H I( H )={(A ⊥ C|B,D), (B ⊥ D|A,C)} Machine Learning Srihari Conditional Independence in Markov Networks • Markov network encodes a set of conditional independencies • Probabilistic influence flows – in undirected paths • Blocked if we condition on intervening nodes – Every path from any node in A to B passes through C – No explaining away • Testing for independence simpler than in directed graphs – Alternative view • Remove all nodes in set C together with all their connecting links • If no paths from A to B then conditional independence holds • Markov blanket • A node is conditionally independent of all other nodes conditioned only on its neighbors 4 Set of nodes C Separates sets A and B Machine Learning Srihari 5 Factorization Properties • Factorization rule corresponds to conditional independence test • Notion of locality needed • Consider two nodes x i and x j not connected by a link – They are conditionally independent given all other nodes in graph • Because there is no direct path between them and • All other paths pass through nodes that are observed and hence those paths are blocked – Expressed as – Where denotes set x of all variables with x i and x j removed • For conditional independence to hold – factorization is such that x i and x j do not appear in the same factor • No path between them other than going through others – leads to graph concept of clique Machine Learning Srihari 6 Graphical Model as Filter • UI is set of distributions that are consistent with set of conditional independence statements read from the graph using graph separation • UF are set of distributions that can be expressed as factorization of the form • Hammersley-Clifford theorem states that UI and UF are identical Machine Learning Srihari Gibbs Distribution and Graph • Let P be a distribution over χ ={X 1 ,..X n } and H is a Markov structure over χ • If P is a Gibbs distribution that factorizes over H , then H is an I-map for...

View
Full
Document