jurafsky&martin_3rdEd_17 (1).pdf

A common alternative is to use supervised machine

Info icon This preview shows pages 431–433. Sign up to view the full content.

View Full Document Right Arrow Icon
A common alternative is to use supervised machine learning. Assuming a train- ing set is available which associates each sentence with the correct semantics, we can train a classifier to map from sentences to intents and domains, and a sequence model to map from sentences to slot fillers. For example given the sentence: I want to fly to San Francisco on Monday afternoon please we might first apply a simple 1-of-N classifier (logistic regression, neural network, etc.) that uses features of the sentence like word N-grams to determine that the domain is AIRLINE and and the intent is SHOWFLIGHT .
Image of page 431

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
432 C HAPTER 28 D IALOG S YSTEMS AND C HATBOTS Next to do slot filling we might first apply a classifier that uses similar features of the sentence to predict which slot the user wants to fill. Here in addition to word unigram, bigram, and trigram features we might use named entity features or features indicating that a word is in a particular lexicon (such as a list of cities, or airports, or days of the week) and the classifer would return a slot name (in this case DESTINATION , DEPARTURE - DAY , and DEPARTURE - TIME ). A second classifier can then be used to determine the filler of the named slot, for example a city classifier that uses N-grams and lexicon features to determine that the filler of the DESTINATION slot is S AN F RANCISCO . An alternative model is to use a sequence model (MEMMs, CRFs, RNNs) to directly assign a slot label to each word in the sequence, following the method used for other information extraction models in Chapter 20 ( Pieraccini et al. 1991 , Raymond and Riccardi 2007 , Mesnil et al. 2015 , Hakkani-T¨ur et al. 2016 ). Once again we would need a supervised training test, with sentences paired with IOB IOB (Inside/Outside/Begin) labels like the following: O O O O O B-DES I-DES O B-DEPTIME I-DEPTIME O I want to fly to San Francisco on Monday afternoon please In IOB tagging we introduce a tag for the beginning (B) and inside (I) of each slot label, and one for tokens outside (O) any slot label. The number of tags is thus 2 n + 1 tags, where n is the number of slots. Any IOB tagger sequence model can then be trained on a training set of such labels. Traditional sequence models (MEMM, CRF) make use of features like word embeddings, word unigrams and bigrams, lexicons (for example lists of city names), and slot transition features (perhaps DESTINATION is more likely to follow ORIGIN than the other way around) to map a user’s utterance to the slots. An MEMM (Chap- ter 10) for example, combines these features of the input word w i , its neighbors within l words w i + l i - l , and the previous k slot tags s i - 1 i - k to compute the most likely slot label sequence S from the word sequence W as follows: ˆ S = argmax S P ( S | W ) = argmax S Y i P ( s i | w i + l i - l , s i - 1 i - k ) = argmax S Y i exp X i w i f i ( s i , w i + l i - l , s i - 1 i - k ) !
Image of page 432
Image of page 433
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern