124.11.lec6

124.11.lec6 - CS 124/LINGUIST 180: From Click to edit...

Info iconThis preview shows pages 1–16. Sign up to view the full content.

View Full Document Right Arrow Icon
Click to edit Master subtitle style 1/10/09 Dan Jurafsky Lecture 6: Hidden Markov Models IP notice:
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Click to edit Master subtitle style 1/10/09 Dan Jurafsky Lecture 6: Hidden Markov Models
Background image of page 2
6/1/11 Outline Markov Chains Hidden Markov Models Three Algorithms for HMMs The Forward Algorithm The Viterbi Algorithm The Baum-Welch (EM Algorithm) Applications: The Ice Cream Task Part of Speech Tagging Biology Gene Finding
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
6/1/11 Definitions A weighted finite-state automaton An FSA with probabilities onthe arcs The sum of the probabilities leaving any arc must sum to one A Markov chain (or observable Markov Model) a special case of a WFST in which the input sequence uniquely determines which states the automaton will go through Markov chains can’t represent inherently ambiguous problems Useful for assigning probabilities to
Background image of page 4
6/1/11 Markov chain for weather
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
6/1/11 Markov chain for words
Background image of page 6
6/1/11 Markov chain = “First-order a set of states Q = q1, q2…qN; the state at time t is qt Transition probabilities: a set of probabilities A = a01a02…an1…ann. Each aij represents the probability of transitioning from state i to state j The set of these is the transition probability matrix A a ij = P ( q t = j | q t - 1 = i ) i , j £ N a ij =1; i £ N j =1 N å
Background image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
6/1/11 Markov chain = “First-order Current state only depends on previous state Markov Assumption : P ( q i | q 1 L q i - 1 ) = P ( q i | q i - 1 )
Background image of page 8
6/1/11 Another representation for start Instead of start state An initial distribution over probability of start states Constraints: π i = P ( q 1 = i ) i £ N j =1 j =1 N å
Background image of page 9

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
6/1/11 The weather figure using pi
Background image of page 10
6/1/11 The weather figure: specific
Background image of page 11

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
6/1/11 Markov chain for weather What is the probability of 4 consecutive warm days? Sequence is warm-warm-warm-warm I.e., state sequence is 3-3-3-3 P(3,3,3,3) = l 3a33a33a33a33 = 0.2 x (0.6)3 = 0.0432
Background image of page 12
6/1/11 How about? Hot hot hot hot Cold hot cold hot What does the difference in these probabilities tell you about the real world weather info encoded in the figure?
Background image of page 13

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
6/1/11 HMM for Ice Cream You are a climatologist in the year 2799 Studying global warming You can’t find any records of the weather in Baltimore, MD for summer of 2008 But you find Jason Eisner’s diary Which lists how many ice-creams Jason ate every date that summer Our job: figure out how hot it was
Background image of page 14
Hidden Markov Model For Markov chains, the output symbols are the same as the states. See hot weather: we’re in state hot But in named-entity or part-of-speech tagging (and speech recognition and other things) The output symbols are words But the hidden states are something else Part-of-speech tags Named entity tags So we need an extension! A
Background image of page 15

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 16
This is the end of the preview. Sign up to access the rest of the document.

This document was uploaded on 06/01/2011.

Page1 / 74

124.11.lec6 - CS 124/LINGUIST 180: From Click to edit...

This preview shows document pages 1 - 16. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online