This preview has intentionally blurred sections. Sign up to view the full version.View Full Document
Unformatted text preview: Part-of-Speech Tagging and Partial Parsing Steven Abney 1996 The initial impetus for the current popularity of statistical methods in com- putational linguistics was provided in large part by the papers on part-of-speech tagging by Church , DeRose , and Garside . In contradiction to com- mon wisdom, these taggers showed that it was indeed possible to carve part- of-speech disambiguation out of the apparently monolithic problem of natural language understanding, and solve it with impressive accuracy. The concensus at the time was that part-of-speech disambiguation could only be done as part of a global analysis, including syntactic analysis, discourse analysis, and even world knowledge. For instance, to correctly disambiguate help in give John help N versus let John help V , one apparently needs to parse the sentences, making reference to the differing subcategorization frames of give and let . Similar examples show that even world knowledge must be taken into account. For instance, off is a preposition in I turned off highway I-90 , but a particle in I turned off my radio , so assigning the correct part of speech in I turned off the spectroroute depends on knowing whether spectroroute is the name of a road or the name of a device. Such examples do demonstrate that the problem of part-of-speech disam- biguation cannot be solved without solving all the rest of the natural-language understanding problem. But Church, DeRose and Garside showed that, even if an exact solution is far beyond reach, a reasonable approximate solution is quite feasible. In this chapter, I would like to survey further developments in part-of-speech disambiguation (‘tagging’). I would also like to consider a question raised by the success of tagging, namely, what piece of the NL-understanding problem we can carve off next. ‘Partial parsing’ is a cover term for a range of different techniques for recovering some but not all of the information contained in a traditional syntactic analysis. Partial parsing techniques, like tagging techniques, aim for reliability and robustness in the face of the vagaries of natural text, by sacrificing completeness of analysis and accepting a low but non-zero error rate. 1 1 Tagging The earliest taggers [35, 51] had large sets of hand-constructed rules for assign- ing tags on the basis of words’ character patterns and on the basis of the tags assigned to preceding or following words, but they had only small lexica, pri- marily for exceptions to the rules. TAGGIT  was used to generate an initial tagging of the Brown corpus, which was then hand-edited. (Thus it provided the data that has since been used to train other taggers .) The tagger described by Garside [56, 34], CLAWS, was a probabilistic version of TAGGIT, and the DeRose tagger improved on CLAWS by employing dynamic programming....
View Full Document
- Spring '09
- Natural Language Processing, Hidden Markov model, partial parsing, Eric Brill, hmm taggers