HMM-handout2

HMM-handout2 - Massachusetts Institute of Technology...

Info icon This preview shows pages 1–7. Sign up to view the full content.

View Full Document Right Arrow Icon
Massachusetts Institute of Technology Department of Electrical Engineering & Computer Science 6.345/HST.728 Automatic Speech Recognition Spring, 2010 4/8/10 Lecture Handouts HMM Training Homework: Hidden Markov Models
Image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
4/2/10 21 6.345/HST.728 Automatic Speech Recognition (2010) HMMs 41 Training an HMM-based Speech Recognition System Larry Gillick 6.345/HST.728 Automatic Speech Recognition (2010) HMMs 42 Training the Acoustic Model via Maximum Likelihood Estimation (MLE) Observe a series of utterances with transcription W and frames Y ( W 1 , Y 1 ), ,( W M , Y M ) Let ϕ represent the (unknown) parameters of the acoustic model Let P ϕ (y | w) be the distribution of the frame sequence given W We shall estimate ϕ as follows : ϕ MLE = arg max ϕ P ϕ i = 1 M ( y i | w i )
Image of page 2
4/2/10 22 6.345/HST.728 Automatic Speech Recognition (2010) HMMs 43 What data do we need? Dictionary of words Pronunciation: string of phonemes Set of recorded utterances Transcription: string of words Initial set of acoustic models Not required but useful 6.345/HST.728 Automatic Speech Recognition (2010) HMMs 44 Some assumptions We have a set of output distributions and transition probabilities (duration distributions) We ʼ ll assume that the states have already been clustered somehow Let ʼ s also assume the output distributions are single Gaussians or mixtures of Gaussians Output distributions f i ( y ) Transition probabilities T ( i , j )
Image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
4/2/10 23 6.345/HST.728 Automatic Speech Recognition (2010) HMMs 45 What must be estimated? For each output distribution, need to estimate a set of mean vectors, a set of (usually diagonal) covariance matrices, and mixture probabilities Must also estimate the corresponding transition probabilities We ʼ ll focus on the output distributions 6.345/HST.728 Automatic Speech Recognition (2010) HMMs 46 How can we estimate these quantities? Align frames in each utterance to the corresponding output distributions Collect all of the frames (from all the training utts) assigned to each output distribution Single Gaussian Compute the means and variances of the assigned frames for each distribution Mixture of Gaussians Estimate the mixture distribution via the EM algorithm, which we ʼ ll return to later
Image of page 4
4/2/10 24 6.345/HST.728 Automatic Speech Recognition (2010) HMMs 47 Two alignment methods Viterbi algorithm Deterministic alignment of training utterances Each frame will be assigned to a unique output distribution * Based on the maximum likelihood state sequence Baum-Welch algorithm (forward-backward algorithm) Probabilistic alignment of training utterances Each frame will be distributed across (possibly multiple) output distributions 6.345/HST.728 Automatic Speech Recognition (2010) HMMs 48 Viterbi Algorithm We wish to determine the most likely (ML) state sequence corresponding to the observed frames y x ML = arg max x P ( x , y ) We can perform this computation via dynamic programming in much the same way that we computed P(y) except, now we shall take maxima instead of summing
Image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
4/2/10 25 6.345/HST.728 Automatic Speech Recognition (2010) HMMs 49
Image of page 6
Image of page 7
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern