SOME SIGNAL What type of article is this Sports Political Dark comedy What

Some signal what type of article is this sports

This preview shows page 61 - 76 out of 109 pages.

“SOME SIGNAL” What type of article is this? Sports Political Dark comedy What entities are covered? And are they covered with positive or negative sentiment? Possible signals ???????? 61
Image of page 61
ASIDE: TERMINOLOGY Documents : groups of free text Actual documents (NYT article, journal paper) Entries in a table Corpus : a collection of documents Terms : individual words Separated by whitespace or punctuation 62
Image of page 62
NLP TASKS Syntax : refers to the grammatical structure of language The rules via which one forms sentences/expressions Semantics : the study of meaning of language John is rectangular and a rainbow. Syntactically correct Semantically meaningless 63
Image of page 63
SYNTAX Tokenization Splitting sentences into tokens Lemmatization/Stemming Turning “organizing” and “organized” into “organiz” Morphological Segmentation How words are formed, and relationships of different parts Easy for English, but other languages are difficult Part-of-speech (POS) Tagging Determine whether a word is a noun/adverb/verb etc. Parsing Create a “parse tree” for a sentence 64
Image of page 64
SEMANTICS: INFORMATION EXTRACTION 65
Image of page 65
SEMANTICS: NAMED ENTITY RECOGNITION Identifying key entities in text 66
Image of page 66
SEMANTICS: SENTIMENT ANALYSIS Deciding if reviews/opinions are positive or negative Heavily used by ad industry today 67
Image of page 67
SEMANTICS: MACHINE TRANSLATION Translating from one language to another Simple substitution of words doesn’t work very well 68 (Image: Google)
Image of page 68
SEMANTICS: MACHINE TRANSLATION 0> how long before the next flight to Alice Springs? 1> wie lang vor dem folgenden Flug zu Alice Springs? 2> how long before the following flight to Alice jump? 3> wie lang vor dem folgenden Flug zu Alice springen Sie? 4> how long before the following flight to Alice do you jump? 5> wie lang, bevor der folgende Flug zu Alice tun, Sie springen? 6> how long, before the following flight to Alice does, do you jump? 7> wie lang bevor der folgende Flug zu Alice tut, tun Sie springen? 8> how long before the following flight to Alice does, do you jump? 9> wie lang, bevor der folgende Flug zu Alice tut, tun Sie springen? 10> how long, before the following flight does to Alice, do do you jump? 11> wie lang bevor der folgende Flug zu Alice tut, Sie tun Sprung? 12> how long before the following flight does leap to Alice, does you? 69 From NLTK Book
Image of page 69
SEMANTICS: QUESTION ANSWERING Answer questions posed a user with specific answers 70 WILLIAM WILKINSON’S “AN ACCOUNT OF THE PRINCIPALITIES OF WALLACHIA AND MOLDOVIA” INSPIRED THIS AUTHOR’S MOST FAMOUS NOVEL Bram Stoker
Image of page 70
SEMANTICS: SPOKEN DIALOGUE SYSTEMS 71
Image of page 71
SEMANTICS: TEXTUAL ENTAILMENT Given two text fragments, determine if one being true entails the other, entails the other's negation, or allows the other to be either true or false 72
Image of page 72
SEMANTICS: DOCUMENT SUMMARIZATION Quite a few tools out there today… e.g., SMMRY 73
Image of page 73
OTHER TASKS Speech Recognition Caption Generation Natural Language Generation Optical Character Recognition Word Sense Disambiguation serve: help with food or drink; hold an office; put ball into play Doing all of these for many different languages 74
Image of page 74
SEMANTICS: TEXT CLASSIFICATION Is it spam?
Image of page 75
Image of page 76

You've reached the end of your free preview.

Want to read all 109 pages?

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture