HMM-handout2

# HMM-handout2 - Massachusetts Institute of Technology...

This preview shows pages 1–7. Sign up to view the full content.

Massachusetts Institute of Technology Department of Electrical Engineering & Computer Science 6.345/HST.728 Automatic Speech Recognition Spring, 2010 4/8/10 Lecture Handouts HMM Training Homework: Hidden Markov Models

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
4/2/10 21 6.345/HST.728 Automatic Speech Recognition (2010) HMMs 41 Training an HMM-based Speech Recognition System Larry Gillick 6.345/HST.728 Automatic Speech Recognition (2010) HMMs 42 Training the Acoustic Model via Maximum Likelihood Estimation (MLE) Observe a series of utterances with transcription W and frames Y ( W 1 , Y 1 ), ,( W M , Y M ) Let ϕ represent the (unknown) parameters of the acoustic model Let P ϕ (y | w) be the distribution of the frame sequence given W We shall estimate ϕ as follows : ϕ MLE = arg max ϕ P ϕ i = 1 M ( y i | w i )
4/2/10 22 6.345/HST.728 Automatic Speech Recognition (2010) HMMs 43 What data do we need? Dictionary of words Pronunciation: string of phonemes Set of recorded utterances Transcription: string of words Initial set of acoustic models Not required but useful 6.345/HST.728 Automatic Speech Recognition (2010) HMMs 44 Some assumptions We have a set of output distributions and transition probabilities (duration distributions) We ʼ ll assume that the states have already been clustered somehow Let ʼ s also assume the output distributions are single Gaussians or mixtures of Gaussians Output distributions f i ( y ) Transition probabilities T ( i , j )

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
4/2/10 23 6.345/HST.728 Automatic Speech Recognition (2010) HMMs 45 What must be estimated? For each output distribution, need to estimate a set of mean vectors, a set of (usually diagonal) covariance matrices, and mixture probabilities Must also estimate the corresponding transition probabilities We ʼ ll focus on the output distributions 6.345/HST.728 Automatic Speech Recognition (2010) HMMs 46 How can we estimate these quantities? Align frames in each utterance to the corresponding output distributions Collect all of the frames (from all the training utts) assigned to each output distribution Single Gaussian Compute the means and variances of the assigned frames for each distribution Mixture of Gaussians Estimate the mixture distribution via the EM algorithm, which we ʼ ll return to later
4/2/10 24 6.345/HST.728 Automatic Speech Recognition (2010) HMMs 47 Two alignment methods Viterbi algorithm Deterministic alignment of training utterances Each frame will be assigned to a unique output distribution * Based on the maximum likelihood state sequence Baum-Welch algorithm (forward-backward algorithm) Probabilistic alignment of training utterances Each frame will be distributed across (possibly multiple) output distributions 6.345/HST.728 Automatic Speech Recognition (2010) HMMs 48 Viterbi Algorithm We wish to determine the most likely (ML) state sequence corresponding to the observed frames y x ML = arg max x P ( x , y ) We can perform this computation via dynamic programming in much the same way that we computed P(y) except, now we shall take maxima instead of summing

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
4/2/10 25 6.345/HST.728 Automatic Speech Recognition (2010) HMMs 49
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern