jurafsky&martin_3rdEd_17 (1).pdf

Bibliographical and historical notes 120 9 hidden

Info icon This preview shows pages 4–7. Sign up to view the full content.

View Full Document Right Arrow Icon
Bibliographical and Historical Notes . . . . . . . . . . . . . . . . . . . . 120 9 Hidden Markov Models 122 9.1 Markov Chains . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 9.2 The Hidden Markov Model . . . . . . . . . . . . . . . . . . . . . 124 9.3 Likelihood Computation: The Forward Algorithm . . . . . . . . . 127 9.4 Decoding: The Viterbi Algorithm . . . . . . . . . . . . . . . . . . 131 9.5 HMM Training: The Forward-Backward Algorithm . . . . . . . . 134 9.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 Bibliographical and Historical Notes . . . . . . . . . . . . . . . . . . . . 140 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141 10 Part-of-Speech Tagging 142 10.1 (Mostly) English Word Classes . . . . . . . . . . . . . . . . . . . 143 10.2 The Penn Treebank Part-of-Speech Tagset . . . . . . . . . . . . . 145 10.3 Part-of-Speech Tagging . . . . . . . . . . . . . . . . . . . . . . . 147 10.4 HMM Part-of-Speech Tagging . . . . . . . . . . . . . . . . . . . 149 10.5 Maximum Entropy Markov Models . . . . . . . . . . . . . . . . . 157 10.6 Bidirectionality . . . . . . . . . . . . . . . . . . . . . . . . . . . 162 10.7 Part-of-Speech Tagging for Other Languages . . . . . . . . . . . . 162 10.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163 Bibliographical and Historical Notes . . . . . . . . . . . . . . . . . . . . 164 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166 11 Formal Grammars of English 168 11.1 Constituency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168 11.2 Context-Free Grammars . . . . . . . . . . . . . . . . . . . . . . . 169 11.3 Some Grammar Rules for English . . . . . . . . . . . . . . . . . . 174 11.4 Treebanks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181 11.5 Grammar Equivalence and Normal Form . . . . . . . . . . . . . . 187 11.6 Lexicalized Grammars . . . . . . . . . . . . . . . . . . . . . . . . 188 11.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 Bibliographical and Historical Notes . . . . . . . . . . . . . . . . . . . . 194 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 12 Syntactic Parsing 197 12.1 Ambiguity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
Image of page 4

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
C ONTENTS 5 12.2 CKY Parsing: A Dynamic Programming Approach . . . . . . . . 199 12.3 Partial Parsing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205 12.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209 Bibliographical and Historical Notes . . . . . . . . . . . . . . . . . . . . 210 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211 13 Statistical Parsing 212 13.1 Probabilistic Context-Free Grammars . . . . . . . . . . . . . . . . 213 13.2 Probabilistic CKY Parsing of PCFGs . . . . . . . . . . . . . . . . 217 13.3 Ways to Learn PCFG Rule Probabilities . . . . . . . . . . . . . . 218 13.4 Problems with PCFGs . . . . . . . . . . . . . . . . . . . . . . . . 220 13.5 Improving PCFGs by Splitting Non-Terminals . . . . . . . . . . . 223 13.6 Probabilistic Lexicalized CFGs . . . . . . . . . . . . . . . . . . . 225 13.7 Probabilistic CCG Parsing . . . . . . . . . . . . . . . . . . . . . . 230 13.8 Evaluating Parsers . . . . . . . . . . . . . . . . . . . . . . . . . . 238 13.9 Human Parsing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239 13.10 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242 Bibliographical and Historical Notes . . . . . . . . . . . . . . . . . . . . 242 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244 14 Dependency Parsing 245 14.1 Dependency Relations . . . . . . . . . . . . . . . . . . . . . . . . 246 14.2 Dependency Formalisms . . . . . . . . . . . . . . . . . . . . . . . 248 14.3 Dependency Treebanks . . . . . . . . . . . . . . . . . . . . . . . 249 14.4 Transition-Based Dependency Parsing . . . . . . . . . . . . . . . 250 14.5 Graph-Based Dependency Parsing . . . . . . . . . . . . . . . . . 261 14.6 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266 14.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267 Bibliographical and Historical Notes . . . . . . . . . . . . . . . . . . . . 268 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269 15 Vector Semantics 270 15.1 Words and Vectors . . . . . . . . . . . . . . . . . . . . . . . . . . 271 15.2 Weighing terms: Pointwise Mutual Information (PMI) . . . . . . . 275 15.3 Measuring similarity: the cosine . . . . . . . . . . . . . . . . . . 279 15.4 Using syntax to define a word’s context . . . . . . . . . . . . . . . 282 15.5 Evaluating Vector Models . . . . . . . . . . . . . . . . . . . . . . 283 15.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284 Bibliographical and Historical Notes . . . . . . . . . . . . . . . . . . . . 284 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285 16 Semantics with Dense Vectors 286 16.1 Dense Vectors via SVD . . . . . . . . . . . . . . . . . . . . . . . 286 16.2 Embeddings from prediction: Skip-gram and CBOW . . . . . . . 290 16.3 Properties of embeddings . . . . . . . . . . . . . . . . . . . . . . 295 16.4 Brown Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . 295 16.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297 Bibliographical and Historical Notes . . . . . . . . . . . . . . . . . . . . 298 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299 17 Computing with Word Senses 300 17.1 Word Senses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300
Image of page 5
6 C ONTENTS 17.2 Relations Between Senses . . . . . . . . . . . . . . . . . . . . . . 303 17.3 WordNet: A Database of Lexical Relations . . . . . . . . . . . . . 305 17.4 Word Sense Disambiguation: Overview . . . . . . . . . . . . . . . 306 17.5 Supervised Word Sense Disambiguation . . . . . . . . . . . . . . 308 17.6 WSD: Dictionary and Thesaurus Methods . . . . . . . . . . . . . 311 17.7 Semi-Supervised WSD: Bootstrapping . . . . . . . . . . . . . . . 314 17.8 Unsupervised Word Sense Induction . . . . . . . . . . . . . . . . 316 17.9 Word Similarity: Thesaurus Methods . . . . . . . . . . . . . . . . 317 17.10 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323 Bibliographical and Historical Notes . . . . . . . . . . . . . . . . . . . . 323 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325 18 Lexicons for Sentiment and Affect Extraction 326 18.1 Available Sentiment Lexicons . . . . . . . . . . . . . . . . . . . . 327 18.2 Semi-supervised induction of sentiment lexicons . . . . . . . . . . 328 18.3 Supervised learning of word sentiment . . . . . . . . . . . . . . . 333 18.4 Using Lexicons for Sentiment Recognition . . . . . . . . . . . . . 337 18.5 Emotion and other classes . . . . . . . . . . . . . . . . . . . . . . 338 18.6 Other tasks: Personality . . . . . . . . . . . . . . . . . . . . . . . 341 18.7 Affect Recognition . . . . . . . . . . . . . . . . . . . . . . . . . . 342 18.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344 Bibliographical and Historical Notes . . . . . . . . . . . . . . . . . . . . 344
Image of page 6

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 7
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern