languagemodeling

Fifth an of futures the an incorporated a a the

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: (the | that ) •  Or maybe P (the | its water is so transparent that) " P (the | transparent that ) Dan Jurafsky Markov Assump1on P ( w1w 2 … w n ) " $ P ( w i | w i # k … w i #1 ) i •  In other words, we approximate each component in the product P ( w i | w1w 2 … w i "1 ) # P ( w i | w i " k … w i "1 ) Dan Jurafsky Simplest case: Unigram model P ( w1w 2 … w n ) " # P ( w i ) i Some automa*cally generated sentences from a unigram model ! fifth, an, of, futures, the, an, incorporated, a, a, the, inflation, most, dollars, quarter, in, is, mass! ! thrift, did, eighty, said, hard, 'm, july, bullish! ! that, or, limited, the! Dan Jurafsky Bigram model " Condi*on on the previous word: P ( w i | w1w 2 … w i "1 ) # P ( w i | w i "1 ) texaco, rose, one, in, this, issue, is, pursuing, growth, in, a, boiler, house, said, mr., gurria, mexico, 's, motion, control, proposal, without, permission, from, five, hundred, fifty, five, yen! ! outside, new, car, parking, lot, of, the, agreement, reached! ! this, would, be, a, record, november! Dan Jurafsky N ­gram models •  We can extend to trigrams, 4 ­grams, 5 ­grams •  In general this is an insufficient model of language •  because language has long ­distance dependencies: “The computer which I had just put into the machine room on the fiIh floor crashed.” •  But we can oIen get away with N ­gram models Language Modeling Introduc*on to N ­grams Language Modeling Es*ma*ng N ­gram Probabili*es Dan Jurafsky Es1ma1ng bigram probabili1es •  The Maximum Likelihood Es*mate count ( w i"1, w i ) P ( w i | w i"1 ) = count ( w i"1 ) ! c ( w i"1, w i ) P ( w i | w i"1 ) = c ( w i"1 ) Dan Jurafsky An example c ( w i"1, w i ) P ( w i | w i"1 ) = c ( w i"1 ) <s> I am Sam </s> <s> Sam I am </s> <s>...
View Full Document

This document was uploaded on 02/14/2014.

Ask a homework question - tutors are online