2420 IS natural language processing Dr. Douglas Metzler
Outline: 1. Course Requirements, Expectations, etc. 2. Motivations 3. Why its Hard 4. Brief History of Linguistics 5. General Approach and Levels of Analysis 6. Formal Models and Algorithms
1. Expectations & Administration Website: www.sis.pitt.edu/~is2420 sign up for listserv (when instructions appear)
Prerequisites: data structures human information processing From these you should be familiar (not necessarily expert) with (some of) the following:
computational theory complexity theory tree traversal, search, backtracking working with algorithms, formalisms cognitive science knowledge representation semantic networks, frames, predicate logic psycholinguisitcs and basic linguistic theory propositional representation speech act/communication theory role of context in understanding Chomsky basic grammar using rewrite rules
2. Motivations: communication window to thought dual coding hypothesis visual: parallel & concrete linguistic: serial & abstract systems: information retrieval interfaces text abstraction text (e.g., report) generation explanation facilities (e.g., for expert systems) intelligent tutoring systems communication direction
3. Why its Hard ambiguity: many to many mapping tool for communication gist implicature (we understand both less and more than the literal words.) dependence on general world knowledge context reasoning & planning conversational conventions complexity: innateness issue brain differentiation ease of learning earlier less conscious less variability cross language regularities developmental regularities timing acquisition patterns critical periods no evidence of deliberate teaching
language vs other means of communication bee dance primate gestures music & dance programming languages Design Features of Language, Hockett 1960's discreteness at each level in spite of fuzzy categories arbitrariness duality of patterning openness effability vs. productivity Abstractions: time displacement abstract relations modalities hypotheticals causality QA, commands, requests
4. Brief History of Approaches to Linguisitics Traditional Grammar normative descriptive not empirical 19th century: Historical Linguisitcs evolution discovery of Sanskrit emphasis on sound lawful change -> scientific analysiss
Early 20th century: Descriptive Linguistics & Anthropological Linguistics de Saussuresystem in unstable equilibrium Endangered Indian Cultures and Languages Boas, Sapir Missionaries Summer Institute of Linguistics 1930's-
Structural Linguistics Bloomfield & Harris influences: behaviorism, logical positivism so: no mental notions concern with observables bottom up emphasis on phonology method: immediate constituent analysis * start with (interpreted) sound stream * discovery procedures -> groupings * each unit tied to sound segment * postpone syntax * outlaw semantics problems: * at best identification of constituents, not explanation * discontinuous elements (can't deal with) * didn't work past phonetics, simple syntax (where abstract explanations required) Behaviorist Theories of Language Skinner, Osgood S-R conditional probabilities
Chomsky * why fear abstract concepts? * immediate constituent analysis misses points, key nature of constituents * need for theory of language, not specimens Theory: * formal nature competence vs performance generative grammar * adequacy criteria observational descriptive explanatory * Rewrite rules A -> B + a noun phrase -> determiner adjective noun <integer> ::= <digit> | <digit><integer> * Chomskian Hierarchy 3. regular or finite state grammar 2. context free 1. context sensitive 0. transformational
Post Chomsky interdisciplinary areas: psycholinguisitcs computational linguistics natural language understanding cognitive science emphasis on semantics pragmatics discourse within generative linguistics (syntax) new theories lexical functional grammar government and binding generalized phrase structure grammar
5. Levels of Analysis Levels of Analysis / Representation of Language phonology morphology syntax semantics pragmatics syntactic (S SUBJ (NP NAME John NUM {3s}) MAIN-V sold TENSE {PAST} VOICE {ACTIVE} OBJ (NP DET the HEAD book NUM {3s}) MODS (PP PREP to POBJ (NP NAME mary NUM {3s}))
world knowledge
logical form [Fillmore, 1968, Case Grammar] (PAST s1 SELL-EVENT [AGENT (NAME p1 PERSON "John")] [THEME (DEF/SING b1 BOOK)] [TO-POSS (NAME p2 PERSON "Mary")]) (PAST go1 TAKE-ACTION [AGENT (NAME p1 PERSON "Jack")] [THEME (NAME p2 PERSON "Mary")] [TO-LOC (DEF/SING b1 PARTY (CELEBRATES b1 HALLOWEEN))])
6. Formal Models and Algorithms state machines formal rule systems logic probability theory machine learning state space search dynamic programming deterministic vs. nondeterministic algorithms procedural vs. declarative models
Regular Expressions standard notation for text sequences (patterns of) sequence disjunction range negation escape from special or nothing repetition /string/ [wW] [1234567890] [1-5] [a-f] [^x] [\.] woodchucks? (zero or one) /a*/ (zero or more) /a+/ (one or more) /./ (any single character)
wildcard anchors sta...