9.4-PartialDirectedCRF

9.4-PartialDirectedCRF - Machine Learning ! ! ! ! ! Srihari...

Info iconThis preview shows pages 1–10. Sign up to view the full content.

View Full Document Right Arrow Icon
Machine Learning Srihari 1 Partially Directed Models and Conditional Random Fields Sargur Srihari srihari@cedar.buffalo.edu
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Machine Learning Srihari Topics • Unifying directed and undirected graphs • Conditional Random Fields – CRF Semantics – CRFs for Text Analysis • Chain Graph Models – Independencies • Summary and Discussion – Bayesian Networks versus Markov Networks 2
Background image of page 2
Machine Learning Srihari Partially Directed Models • Can unify directed/undirected dependencies • Conditional Random Fields have dependency on some subset of variables • They can be generalized to chain graphs – Network in which undirected components depend upon each other in a directed fashion 3
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Machine Learning Srihari Conditional Random Fields • Use Markov Network formalism (to represent a joint distribution over X) to represent a conditional distribution P(Y|X) Y is a set of target variables X is a set of observed variables • In the case of MNs it is called a CRF • Has an analog in directed graphical models – Conditional Bayesian Networks 4
Background image of page 4
Machine Learning Srihari CRF Representation and Semantics • CRF nodes correspond to Y U X Y are target variables and X are observed variables • Parameterized as ordinary Markov Network – Set of factors Φ 1 (D 1 ),. . Φ m (D m ) • Can be encoded as a log-linear model • Viewed as encoding a set of factors • Instead of P(Y,X) view it as representing P(Y|X) • To naturally represent a conditional distribution – Disallow potentials involving only variables in X 5
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Machine Learning Srihari CRF Definition • A CRF is an undirected graph H whose nodes correspond to X U Y • Network is annotated with a set of factors • Network encodes a conditional distribution as • Two variables in H are connected by an edge whenever they appear in the scope of a factor P ( Y | X ) = 1 Z ( X ) P ( Y , X ) P ( Y , X ) = φ i ( D i ) i = 1 m Z ( X ) = P ( Y , X ) Y 1 ( D 1 ),. . m ( D m ) such that D i X Partition function Is now a function of X Where Z(X) is the marginal distribution of X and P(Y,X) is the joint distribution Joint distribution is a product of factors
Background image of page 6
Machine Learning Srihari CRF vs Gibbs Distribution • Different normalization in partition function Z(X) • CRF induces a different value in the partition function for every assignment x to X • Difference is denoted graphically by grayed out nodes 7
Background image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Machine Learning Srihari Example of CRF • CRF over Y={Y 1 ,..Y k } and X={X 1 ,..X k } • Edges are Y i —Y i+1 and Y i —X i 8 P ( Y | X ) = 1 Z ( X ) P ( Y , X ) P ( Y , X ) = φ i ( Y i , Y i + 1 ) i = 1 k 1 i ( Y i , X i ) i = 1 k Z ( X ) = P ( Y , X ) Y Observed Feature Variables Called sequence labeling
Background image of page 8
Machine Learning Srihari CRF as Partially Directed Graph • CRF defines a conditional distribution of Y on X • Can be viewed as one with undirected component over Y which has X as parents 9
Background image of page 9

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 10
This is the end of the preview. Sign up to access the rest of the document.

This document was uploaded on 02/25/2012.

Page1 / 31

9.4-PartialDirectedCRF - Machine Learning ! ! ! ! ! Srihari...

This preview shows document pages 1 - 10. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online