La green normalize 12 12 0 rows to sum house 14 12 14

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: abili$es Kevin Knight’s example … la maison … la maison bleue … la fleur … … the house … the blue house … the flower … •  EM reveals inherent hidden structure! •  We can now es1mate parameters from aligned corpus: p(la|the) = 0.453 p(le|the) = 0.334 p(maison|house) = 0.876 p(bleu|blue) = 0.563 Dan Jurafsky The EM Algorithm for Word Alignment 1.  Ini1alize the model, typically with uniform distribu1ons 2.  Repeat E Step: Use the current model to compute the probability of all possible alignments of the training data M Step: Use these alignment probability es1mates to re ­es1mate values for all of the parameters. un1l converge (i.e., parameters no longer change) 34 Dan Jurafsky Example EM Trace for Model 1 Alignment •  Simplified version of Model 1 (No NULL word, and subset of alignments: ignore alignments for which English word aligns with no foreign word) (ignoring a constant here) •  E ­step J P( A, F | E ) = ! t ( f j ea j ) j =1 •  Normalize to get probability of an alignment: J P( A E, F ) = 35 P( A, F E ) !A P( A, F E ) " t ( f j ea ) j = j =1 J ! A " t ( f j ea ) j j =1 Dan Jurafsky Sample EM Trace for Alignment: E step (IBM Model 1 with no NULL Genera$on) J P( A, F | E ) = ! t ( f j ea j ) Training Corpus j =1 green house the house casa verde la casa P( A, F E ) P( A E, F ) = ! A P( A, F E ) verde casa la 1/3 1/3 green 1/3 Assume uniform Transla1on house 1/3 1/3 1/3 ini1al probabili1es Probabili1es the 1/3 1/3 1/3 Compute Alignment Probabili1es P(A, F | E) Normalize to get P(A | F, E) green house green house casa verde casa verde the house the house la casa la casa 1/3 X 1/3 = 1/9 1/3 X 1/3 = 1/9 1/3 X 1/3 = 1/9 1/3 X 1/3 = 1/9 1/ 9 1 = 2/9 2 1/ 9 1 = 2/9 2 1/ 9 1 = 2/9 2 1/ 9 1 = 2/9 2 Dan Jurafsky EM example con$nued: M step green house green house the house the house c...
View Full Document

This document was uploaded on 02/14/2014.

Ask a homework question - tutors are online