413 CMPT Computational Linguistics
Anoop Sarkar
http://www.cs.sfu.ca/~anoop
3/21/07
1
Context-free Grammars
Set of rules by which valid sentences can be constructed. Example:
Sentence ! Noun Verb Object Noun ! trees | parsers Verb ! are | grow Object ! on Noun | Adjective Adjective ! slowly | interesting
What strings can Sentence derive? Syntax only no semantic checking
3/21/07 2
Derivations of a CFG
parsers grow on trees parsers grow on Noun parsers grow Object parsers Verb Object Noun Verb Object Sentence
3
3/21/07
Derivations and parse trees
Sentence Noun Verb Object Noun parsers
3/21/07
grow
on
trees
4
Arithmetic Expressions
E!E+E E!E*E E!(E) E!-E E ! id
3/21/07
5
Leftmost derivations for id + id * id
E!E+E E!E*E E!(E) E!-E E ! id E"E+E " id + E " id + E * E " id + id * E " id + id * id
E E id + E
E id
*
E id
6
3/21/07
Leftmost derivations for id + id * id
E!E+E E!E*E E!(E) E!-E E ! id E"E*E "E+E*E " id + E * E " id + id * E " id + id * id
E E E id + E id * E id
3/21/07
7
Rightmost derivation for id + id * id
E!E+E E!E*E E!(E) E!-E E ! id E"E+E "E+E*E " E + E * id " E + id * id " id + id * id
E E id + E
E id
*
E id
8
3/21/07
Rightmost derivation for id + id * id
E!E+E E!E*E E!(E) E!-E E ! id E"E*E " E * id " E + E * id " E + id * id " id + id * id
E E E id + E id * E id
3/21/07
9
Parsing - Roadmap
Parser is a decision procedure: builds a parse tree Top-down vs. bottom-up Recursive-descent with backtracking Bottom-up parsing (CKY) Shift-reduce parsing Combining top-down and bottom-up: Earley parsing
3/21/07 10
Top-Down vs. Bottom Up
Grammar: S ! A B A!c|# B ! cbB | ca Top-Down/leftmost S " AB " cB " ccbB " ccbca
3/21/07
Input String: ccbca
Bottom-Up/rightmost A!c B!ca B!cbB S!AB
11
S!AB ccbca $ Acbca A!c $ AcbB B!cbB $ AB B!ca $S
Top-Down: Backtracking
S!AB A!c|# B ! cbB | ca True/False S "* cbca?
S AB cB B #B cbB bB B cbB bB ca a cbca cbca cbca bca cbca cbca bca ca ca a ca a try S!AB try A!c match c dead-end, try A!# try B!cbB match c match b try B!cbB match c dead-end, try B!ca match c match a, Done!
12
3/21/07
Transition Diagram
S ! cAa A ! cB | B
S: c A a
A:
c B
B
B ! bcB | #
3/21/07
B:
b
c #
B
13
Bottom-up parsing overview
Start from terminal symbols, search for a path to the start symbol Apply shift and reduce actions: postpone decisions LR parsing:
L: left to right parsing R: rightmost derivation (in reverse or bottom-up)
Useful for deterministic parsing (e.g. in compilers for programming languages)
3/21/07
14
Rightmost derivation for id + id * id
E!E+E E!E*E E!(E) E!-E E ! id E"E*E " E * id " E + E * id " E + id * id " id + id * id
reduce with E ! id shift
3/21/07
15
Ambiguity
Grammar is ambiguous if more than one parse tree is possible for some sentences Examples in English:
Two sisters reunited after 18 years in checkout counter
It is undecidable to check using an algorithm whether a grammar is ambiguous
3/21/07
16
Parsing CFGs
Consider the problem of parsing with arbitrary CFGs For any input string, the parser has to produce a parse tree The simpler problem: print yes if the input string is generated by the grammar, print no otherwise This problem is called recognition
3/21/07 17
CKY Recognition Algorithm
The Cocke-Kasami-Younger algorithm As we shall see it runs in time that is polynomial in the size of the input It takes space polynomial in the size of the input Remarkable fact: it can find all possible parse trees (exponentially many) in polynomial time
3/21/07 18
Chomsky Normal Form
Before we can see how CKY works, we need to convert the input CFG into Chomsky Normal Form CNF is one of many grammar transformations that preserve the language CNF means that the input CFG G is converted to a new CFG G' in which all rules are of the form:
A!BC A!a
3/21/07 19
Epsilon Removal
First step, remove epsilon rules
A!BC C!#|CD|a D!b B!b
After #-removal:
A ! B | B C D | B a | BC C!D|CDD|aD|CD|a D!b B!b
3/21/07 20
Removal of Chain Rules
Second step, remove chain rules
A!BC|CDC C!D|a D!d B!b
After removal of chain rules:
A!Ba|BD|aDa|aDD|DDa|DDD D!d B!b
3/21/07 21
Eliminate terminals RHS
from Third step, remove terminals from the rhs of rules
A!BaCd
After removal of terminals from the rhs:
A ! B N1 C N2 N1 ! a N2 ! d
3/21/07
22
Binarize RHS with Nonterminals
Fourth step, convert the rhs of each rule to have two non-terminals
A ! B N1 C N2 N1 ! a N2 ! d
After converting to binary form:
A ! B N3 N3 ! N1 N4 N4 ! C N2
3/21/07
N1 ! a N2 ! d
23
CKY algorithm
We will consider the working of the algorithm on an example CFG and input string Example CFG:
S!AX|YB X!AB|BA A!a B!a
3/21/07
Y!BA
Example input string: aaa
24
CKY Algorithm
0 0 1 2
A, B A!a B!a
1
2
X, Y X!AB|BA Y!BA A, B A!a B!a
3
S
S ! A(0,1) X(1,3) S ! Y(0,2) B(2,3) X, Y X!AB|BA Y!BA A, B A!a B!a
a
3/21/07
a
a
25
Parse trees
S Y B a A a a a B A A a S X B a a A B a S X A a
3/21/07
26
CKY Algorithm
Input string input of size n Create a 2D table chart of size n2 for i=0 to n-1 chart[i][i+1] = A if there is a rule A ! a and input[i]=a for j=2 to N for i=j-2 downto 0 for k=i+1 to j-1 chart[i][j] = A if there is a rule A ! B C and chart[i][k] = B and chart[k][j] = C return yes if chart[0][n] has the start symbol else return no
3/21/07 27
CKY algorithm summary
Parsing arbitrary CFGs For the CKY algorithm, the time complexity is O(|G|2 n3) The space requirement is O(n2) The CKY algorithm handles arbitrary ambiguous CFGs All ambiguous choices are stored in the chart For compilers we consider parsing algorithms for CFGs that do not handle ambiguous grammars
3/21/07 28
Parsing - Summary
Parsing arbitrary CFGs: O(n3) time complexity Top-down vs. bottom-up
Recursive-descent parsing Shift-reduce parsing
Earley parsing Ambiguous grammars result in parser output with multiple parse trees for a single input string
3/21/07
29
Parsing - Additional Results
O(n2) time complexity for linear grammars
All rules are of the form S ! aSb or S ! a Reason for O(n2) bound is the linear grammar normal form: A ! aB, A ! Ba, A ! B, A ! a
Left corner parsers
extension of top-down parsing to arbitrary CFGs
Earley's parsing algorithm
O(n3) worst case time for arbitrary CFGs just like CKY O(n2) worst case time for unambiguous CFGs O(n) for specific unambiguous grammars 3/21/07 (e.g. S ! aSa | bSb | #) 30
Non-CF Languages
3/21/07
31
CF Languages
3/21/07
32
Context-free languages and Pushdown Automata
Recall that for each regular language there was an equivalent finite-state automaton The FSA was used as a recognizer of the regular language For each context-free language there is also an automaton that recognizes it: called a pushdown automaton (...