16-AnalyzingTextData.pdf - Analyzing Text Data Introduction to Computational Thinking and Data Science Anna Farzindar [email protected] Many Interesting

16-AnalyzingTextData.pdf - Analyzing Text Data Introduction...

This preview shows page 1 - 18 out of 53 pages.

Analyzing Text Data Introduction to Computational Thinking and Data Science Anna Farzindar [email protected]
Image of page 1
Many Interesting Problems Involve Text Data u Detecting important trends from tweets u Showing relevant ads based on the text of a blog u Organizing news from different newspapers and web sites u Analyzing on-line communities of interest u Automatic Summarization § Summarization of news articles or legal texts § Summarizing customer ratings and comments. 2
Image of page 2
Simple Text Processing Workflow: Encryption 3
Image of page 3
Simple Text Processing Workflow: Comparing Two Documents 4
Image of page 4
Today’s Topics 1. What is text data 2. Text on the Web 3. Text pre-processing 4. Processing document collections 5
Image of page 5
What Is Text Data? 6
Image of page 6
Is This Text? Handwritten Medical Records 7
Image of page 7
Is This Text? Transcribed Notes 8
Image of page 8
Is this Text? Laboratory Notebook 9
Image of page 9
Is This Text? An Old Manuscript 10
Image of page 10
Turning a Manuscript into Text: Encryption Alphabets 11
Image of page 11
Text on the Web 12
Image of page 12
Common Text Sources for Analysis Web pages Social media (e.g., tweets) News articles Emails 13
Image of page 13
Text on the Web u Rarely plain text u Web pages are often not plain text, for example: u UNICODE u HTML markup u PHP 14
Image of page 14
Encoding Languages: UNICODE u Alphabets in different languages are represented using a code called UNICODE u Example: For the Arabic character use the code 0681 15
Image of page 15
HTML Markup “View page source” 16
Image of page 16
HTML Markup <TITLE>Yolanda Gil's Homepage</TITLE> <link rel="meta" type="application/rdf+xml" title="FOAF” href=" foaf.rdf " /> <H1> <a href=" foaf.rdf#me ">Yolanda Gil</a></H1> I am <b>Principal Investigator and Project Leader</b> of the <b>
Image of page 17
Image of page 18

You've reached the end of your free preview.

Want to read all 53 pages?

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture

Stuck? We have tutors online 24/7 who can help you get unstuck.
A+ icon
Ask Expert Tutors You can ask You can ask You can ask (will expire )
Answers in as fast as 15 minutes