More In Depth...
Grammars Parsing
(Slides copied liberally from Ruth Anderson, Hal Perkins and others)
04/19/09
CS415 - Fall 2005
1
Parsing
The syntax of most programming languages can be specified by a context-free grammar (CGF) Parsing: Given a grammar G and a sentence w in L(G ), traverse the derivation (parse tree) for w in some standard order and do something useful at each node
The tree might not be produced explicitly, but the control flow of a parser corresponds to a traversal
04/19/09
CS415 - Fall 2005
2
Old Example
program program statement assignStmt id expr int
G
program ::= statement | program statement statement ::= assignStmt | ifStmt assignStmt ::= id = expr ; ifStmt ::= if ( expr ) stmt expr ::= id | int | expr + expr Id ::= a | b | c | i | j | k | n | x | y | z int ::= 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9
statement ifStmt expr expr expr id int statement assignStmt id expr int
w
04/19/09
a = 1 ; if
( a
+
1 )
b = 2 ;
3
CS415 - Fall 2005
"Standard Order"
For practical reasons we want the parser to be deterministic (no backtracking), and we want to examine the source program from left to right.
(i.e., parse the program in linear time in the order it appears in the source file)
04/19/09
CS415 - Fall 2005
4
Common Orderings
Top-down
Start with the root Traverse the parse tree depth-first, left-to-right (leftmost derivation) LL(k)
Bottom-up
Start at leaves and build up to the root
Effectively a rightmost derivation in reverse(!)
LR(k) and subsets (LALR(k), SLR(k), etc.)
04/19/09
CS415 - Fall 2005
5
Top-down vs. bottom-up
Consider the grammar (Scott, p. 49)
id_list -> id id_list_tail id_list_tail -> , id id_list_tail id_list_tail -> ;
And input text:
A, B, C;
04/19/09
CS415 - Fall 2005
6
04/19/09
CS415 - Fall 2005
7
"Something Useful"
At each point (node) in the traversal, perform some semantic action
Construct nodes of full parse tree (rare) Construct abstract syntax tree (common) Construct linear, lower-level representation (more common in later parts of a modern compiler) Generate target code on the fly (1-pass compiler; not common in production compilers can't generate very good code in one pass)
04/19/09
CS415 - Fall 2005
8
Context-Free Grammars
Formally, a grammar G where is a tuple <N,,P,S>
N a finite set of non-terminal symbols a finite set of terminal symbols P a finite set of productions
A subset of N (N U )*
S the start symbol, a distinguished element of N
If not specified otherwise, this is usually assumed to be the non-terminal on the left of the first production
04/19/09
CS415 - Fall 2005
9
Standard Notations
a, b, c w, x, y, z A, B, C X, Y, Z , , A or A ::=
04/19/09
elements of elements of * elements of N elements of N U elements of (N U )*
if <A, > in P
CS415 - Fall 2005 10
Derivation Relations (1)
A =>
derives
iff A ::= in P
A =>* w if there is a chain of productions starting with A that generates w
transitive closure
04/19/09
CS415 - Fall 2005
11
Derivation Relations (2)
w A =>lm w
derives leftmost
iff A ::= in P
A w =>rm w iff A ::= in P
derives rightmost
We will only be interested in leftmost and rightmost derivations not random orderings
04/19/09 CS415 - Fall 2005 12
Languages
For A in N, L(A) = { w | A =>* w } If S is the start symbol of grammar G, define L(G ) = L(S )
04/19/09
CS415 - Fall 2005
13
Reduced Grammars
Grammar G is reduced iff for every production A ::= in G there is a derivation S =>* x A z => x z =>* xyz
i.e., no production is useless
Convention: will we use only reduced grammars
04/19/09 CS415 - Fall 2005 14
Ambiguity
Grammar G is unambiguous iff every w in L(G ) has a unique leftmost (or rightmost) derivation
Fact: unique leftmost or unique rightmost implies the other
A grammar without this property is ambiguous
Note that other grammars that generate the same language may be unambiguous
We need unambiguous grammars for parsing
04/19/09
CS415 - Fall 2005
15
Example: Ambiguous Grammar for Arithmetic Expressions
expr ::= expr + expr | expr - expr | expr * expr | expr / expr | int int ::= 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 Exercise: show that this is ambiguous
How? Show two different leftmost or rightmost derivations for the same string Equivalently: show two different parse trees for the same string
04/19/09 CS415 - Fall 2005 16
Example (cont)
Give two leftmost derivations of 2+3*4 and show the parse tree
expr ::= expr + expr | expr - expr | expr * expr | expr / expr | int int ::= 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9
04/19/09
CS415 - Fall 2005
17
Another example
Give two different derivations of 5+6+7
expr ::= expr + expr | expr - expr | expr * expr | expr / expr | int int ::= 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9
04/19/09
CS415 - Fall 2005
18
What's going on here?
The grammar has no notion of precedence or associatively Solution
Create a non-terminal for each level of precedence Isolate the corresponding part of the grammar Force the parser to recognize higher precedence sub-expressions first
04/19/09
CS415 - Fall 2005
19
Classic Expression Grammar
expr ::= expr + term | expr term | term term ::= term * factor | term / factor | factor factor ::= int | ( expr ) int ::= 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7
04/19/09
CS415 - Fall 2005
20
Check: Derive 2 + 3 * 4
expr ::= expr + term | expr term | term term ::= term * factor | term / factor | factor factor ::= int | ( expr ) int ::= 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7
CS415 - Fall 2005 21
04/19/09
Check: Derive 5 + 6 + 7
expr ::= expr + term | expr term | term term ::= term * factor | term / factor | factor factor ::= int | ( expr ) int ::= 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7
CS415 - Fall 2005
Note interaction between le...