PA1
Announcements
Due today at midnight
README, test case
Your name(s)!
Error Handling
Syntax-Directed Translation
Recursive Descent Parsing
WA1
Due today at 5pm
Lecture 6
PA2
Assigned today
WA2
Assigned Tuesday
Prof. Aiken CS 143 Lecture 6
2
Prof. Aiken CS 143 Lecture 6
1
Outline
Error Handling
Extensions of CFG for parsing
Purpose of the compiler is
Precedence declarations
Error handling
Semantic actions
To detect non-valid programs
To translate the valid ones
Many kinds of possible errors (e.g. in C)
Error kind
Constructing a parse tree
Lexical
Syntax
Semantic
Correctness
Recursive descent
Prof. Aiken CS 143 Lecture 6
3
Example
Detected by
$
x *%
int x; y = x(3);
your favorite program
Lexer
Parser
Type checker
Tester/User
Prof. Aiken CS 143 Lecture 6
Syntax Error Handling
Approaches to Syntax Error Recovery
Error handler should
4
From simple to complex
Report errors accurately and clearly
Recover from an error quickly
Not slow down compilation of valid code
Panic mode
Error productions
Automatic local or global correction
Good error handling is not easy to achieve
Prof. Aiken CS 143 Lecture 6
Not all are supported by all parser generators
5
Prof. Aiken CS 143 Lecture 6
6
1
Error Recovery: Panic Mode
Syntax Error Recovery: Panic Mode (Cont.)
Simplest, most popular method
Consider the erroneous expression
When an error is detected:
Panic-mode recovery:
(1 + + 2) + 3
Discard tokens until one with a clear role is found
Continue from there
Such tokens are called synchronizing tokens
Skip ahead to next integer and then continue
Bison: use the special terminal error to
describe how much input to skip
E int | E + E | ( E ) | error int | ( error )
Typically the statement or expression terminators
Prof. Aiken CS 143 Lecture 6
7
Prof. Aiken CS 143 Lecture 6
8
Syntax Error Recovery: Error Productions
Error Recovery: Local and Global Correction
Idea: specify in the grammar known common mistakes
Idea: find a correct nearby program
Essentially promotes common errors to alternative
syntax
Disadvantages:
Example:
Write 5 x instead of 5 * x
Add the production E | E E
Disadvantage
Complicates the grammar
Prof. Aiken CS 143 Lecture 6
Try token insertions and deletions
Exhaustive search
9
Hard to implement
Slows down parsing of correct programs
Nearby is not necessarily the intended program
Not all tools support it
Prof. Aiken CS 143 Lecture 6
Syntax Error Recovery: Past and Present
Abstract Syntax Trees
Past
10
So far a parser traces the derivation of a
sequence of tokens
Slow recompilation cycle (even once a day)
Find as many errors in one cycle as possible
Researchers could not let go of the topic
The rest of the compiler needs a structural
representation of the program
Present
Quick recompilation cycle
Users tend to correct one error/cycle
Complex error recovery is less compelling
Panic-mode seems enough
Prof. Aiken CS 143 Lecture 6
Abstract syntax trees
Like parse trees but ignore some details
Abbreviated as AST
11
Prof. Aiken CS 143 Lecture 6
12
2
Abstract Syntax Tree. (Cont.)
Example of Parse Tree
Consider the grammar
E
E int | ( E ) | E + E
E
And the string
5 + (2 + 3)
int5
After lexical analysis (a list of tokens)
(
+
int2
During parsing we build a parse tree
13
Prof. Aiken CS 143 Lecture 6
E
E
int5 + ( int2 + int3 )
Example of Abstract Syntax Tree
Traces the operation
of the parser
E
+
Does capture the
nesting structure
)
But too much info
E
int3
Parentheses
Single-successor
nodes
Prof. Aiken CS 143 Lecture 6
14
Semantic Actions
This is what well use to construct ASTs
PLUS
PLUS
5
Each grammar symbol may have attributes
2
For terminal symbols (lexical tokens) attributes can
be calculated by the lexer
3
Also captures the nesting structure
But abstracts from the concrete syntax
Each production may have an action
=> more compact and easier to use
An important data structure in a compiler
Prof. Aiken CS 143 Lecture 6
Written as: X Y1 Yn
{ action }
That can refer to or compute symbol attributes
15
Prof. Aiken CS 143 Lecture 6
Semantic Actions: An Example
Semantic Actions: An Example (Cont.)
Consider the grammar
16
String: 5 + (2 + 3)
Tokens: int5 + ( int2 + int3 )
E int | E + E | ( E )
For each symbol X define an attribute X.val
Productions
For terminals, val is the associated lexeme
For non-terminals, val is the expressions value (and is
computed from values of subexpressions)
E E1 + E2
E1 int5
E2 ( E3)
E3 E4 + E5
E4 int2
E5 int3
We annotate the grammar with actions:
E int
| E1 + E2
| ( E1 )
{ E.val = int.val }
{ E.val = E1.val + E2.val }
{ E.val = E1.val }
Prof. Aiken CS 143 Lecture 6
17
Equations
E.val = E1.val + E2.val
E1.val = int5.val = 5
E2.val = E3.val
E3.val = E4.val + E5.val
E4.val = int2.val = 2
E5.val = int3.val = 3
Prof. Aiken CS 143 Lecture 6
18
3
Semantic Actions: Notes
Dependency Graph
E
Semantic actions specify a system of
equations
E1
Order of resolution is not specified
Example:
int5
E3.val = E4.val + E5.val
Must compute E4.val and E5.val before E3.val
We say that E3.val depends on E4.val and E5.val
Prof. Aiken CS 143 Lecture 6
(
5
E3
int2
)
+
+
E4
E5
int3
2
19
Evaluating Attributes
3
Prof. Aiken CS 143 Lecture 6
20
Dependency Graph
An attribute must be computed after all its
successors in the dependency graph have been
computed
In previous example attributes can be computed
bottom-up
Such an order exists when there are no cycles
Cyclically defined attributes are not legal
E
E1 5
int5
21
10
E2
+
(
5
E4
int2
Prof. Aiken CS 143 Lecture 6
E2
+
The parser must find the order of evaluation
Each node labeled E
has one slot for the val
attribute
Note the dependencies
+
5
E3
5
+
2
2
)
E5
3
int3
3
Prof. Aiken CS 143 Lecture 6
Semantic Actions: Notes (Cont.)
Inherited Attributes
Synthesized attributes
22
Another kind of attribute
Calculated from attributes of descendents in the
parse tree
E.val is a synthesized attribute
Can always be calculated in a bottom-up order
Grammars with only synthesized attributes
are called S-attributed grammars
Calculated from attributes of parent and/or
siblings in the parse tree
Example: a line calculator
Most common case
Prof. Aiken CS 143 Lecture 6
23
Prof. Aiken CS 143 Lecture 6
24
4
A Line Calculator
Attributes for the Line Calculator
Each line contains an expression
E int | E + E
Each line is terminated with the = sign
LE= | +E=
Each E has a synthesized attribute val
Calculated as before
Each L has an attribute val
In second form the value of previous line is
used as starting value
A program is a sequence of lines
P |PL
LE=
| +E=
We need the value of the previous line
We use an inherited attribute L.prev
25
Prof. Aiken CS 143 Lecture 6
Attributes for the Line Calculator (Cont.)
Example of Inherited Attributes
P
The value of its last line
P
{ P.val = 0 }
{ P.val = L.val;
| P1 L
L.prev = P1.val }
Each L has an inherited attribute prev
L.prev is inherited from sibling P1.val
0
0
27
+
5
0
E3
E4
2
int2
2
+
=
5
E5
3
int3
3
Prof. CS Aiken 143 Lecture 6
E3
+
2
=
+
E5
int3
3
All can be
computed in
depth-first
order
Prof. Aiken CS 143 Lecture 6
28
Semantic Actions: Notes (Cont.)
prev inherited
+
0
E4
val synthesized
L
+
prev inherited
Example of Inherited Attributes
P
L
int2
5
val synthesized
P
Example
Prof. Aiken CS 143 Lecture 6
26
Prof. Aiken CS 143 Lecture 6
Each P has a synthesized attribute val
P
{ L.val = E.val }
{ L.val = E.val + L.prev }
All can be
computed in
depth-first
order
29
Semantic actions can be used to build ASTs
And many other things as well
Also used for type checking, code generation,
Process is called syntax-directed translation
Substantial generalization over CFGs
Prof. Aiken CS 143 Lecture 6
30
5
Constructing An AST
Constructing a Parse Tree
We first define the AST data type
We define a synthesized attribute ast
Supplied by us for the project
Values of ast values are ASTs
We assume that int.lexval is the value of the
integer lexeme
Computed using semantic actions
Consider an abstract tree type with two
constructors:
mkleaf(n)
mkplus(
,
T1
n
=
)
=
E int
| E1 + E2
| ( E1 )
PLUS
T2
T1
E.ast = mkleaf(int.lexval)
E.ast = mkplus(E1.ast, E2.ast)
E.ast = E1.ast
T2
31
Prof. Aiken CS 143 Lecture 6
Prof. Aiken CS 143 Lecture 6
Parse Tree Example
Summary
Consider the string int5 + ( int2 + int3 )
A bottom-up evaluation of the ast attribute:
32
We can specify language syntax using CFG
E.ast = mkplus(mkleaf(5),
mkplus(mkleaf(2), mkleaf(3))
A parser will answer whether s L(G)
and will build a parse tree
which we convert to an AST
and pass on to the rest of the compiler
PLUS
PLUS
5
2
3
33
Prof. Aiken CS 143 Lecture 6
Prof. Aiken CS 143 Lecture 6
Intro to Top-Down Parsing: The Idea
Terminals are seen in order of
appearance in the token
stream:
t2 t5 t6 t8 t9
Prof. Aiken CS 143 Lecture 6
Recursive Descent Parsing
Consider the grammar
The parse tree is constructed
From the top
From left to right
34
E T |T + E
T int | int * T | ( E )
1
3
t2
4
t9
Token stream is: ( int5 )
7
Start with top-level non-terminal E
t5
t6
t8
35
Try the rules for E in order
Prof. Aiken CS 143 Lecture 6
36
6
Recursive Descent Parsing
Recursive Descent Parsing
E T |T + E
T int | int * T | ( E )
E T |T + E
T int | int * T | ( E )
E
E
T
( int5 )
( int5 )
Prof. Aiken CS 143 Lecture 6
37
Prof. Aiken CS 143 Lecture 6
Recursive Descent Parsing
Recursive Descent Parsing
E T |T + E
T int | int * T | ( E )
38
E T |T + E
T int | int * T | ( E )
E
E
T
T
Mismatch: int is not ( !
Backtrack
int
( int5 )
( int5 )
Prof. Aiken CS 143 Lecture 6
39
Prof. Aiken CS 143 Lecture 6
Recursive Descent Parsing
Recursive Descent Parsing
E T |T + E
T int | int * T | ( E )
40
E T |T + E
T int | int * T | ( E )
E
E
T
T
int *
Mismatch: int is not ( !
T Backtrack
( int5 )
( int5 )
Prof. Aiken CS 143 Lecture 6
41
Prof. Aiken CS 143 Lecture 6
42
7
Recursive Descent Parsing
Recursive Descent Parsing
E T |T + E
T int | int * T | ( E )
E T |T + E
T int | int * T | ( E )
E
T
(
E
T
E
)
Match! Advance input.
( int5 )
(
E
)
( int5 )
Prof. Aiken CS 143 Lecture 6
43
Prof. Aiken CS 143 Lecture 6
Recursive Descent Parsing
Recursive Descent Parsing
E T |T + E
T int | int * T | ( E )
44
E T |T + E
T int | int * T | ( E )
E
T
(
E
T
E
(
)
T
( int5 )
45
)
Match! Advance input.
T
( int5 )
Prof. Aiken CS 143 Lecture 6
E
int
Prof. Aiken CS 143 Lecture 6
Recursive Descent Parsing
Recursive Descent Parsing
E T |T + E
T int | int * T | ( E )
46
E T |T + E
T int | int * T | ( E )
E
T
(
( int5 )
E
T
E
)
Match! Advance input.
T
int
Prof. Aiken CS 143 Lecture 6
(
( int5 )
47
E
)
End of input, accept.
T
int
Prof. Aiken CS 143 Lecture 6
48
8
A Recursive Descent Parser. Preliminaries
A Recursive Descent Parser (2)
Let TOKEN be the type of tokens
Define boolean functions that check the token
string for a match of
Special tokens INT, OPEN, CLOSE, PLUS, TIMES
Let the global next point to the next token
Prof. Aiken CS 143 Lecture 6
A given token terminal
bool term(TOKEN tok) { return *next++ == tok; }
The nth production of S:
bool Sn() { }
Try all productions of S:
bool S() { }
49
50
Prof. Aiken CS 143 Lecture 6
A Recursive Descent Parser (3)
A Recursive Descent Parser (4)
For production E T
Functions for non-terminal T
bool T1() { return term(INT); }
bool T2() { return term(INT) && term(TIMES) && T(); }
bool T3() { return term(OPEN) && E() && term(CLOSE); }
bool E1() { return T(); }
For production E T + E
bool E2() { return T() && term(PLUS) && E(); }
For all productions of E (with backtracking)
bool E() {
TOKEN *save = next;
return (next = save, E1())
|| (next = save, E2()); }
Prof. Aiken CS 143 Lecture 6
bool T() {
TOKEN *save = next;
return (next = save, T1())
|| (next = save, T2())
|| (next = save, T3()); }
51
Recursive Descent Parsing. Notes.
Example
To start the parser
E T |T + E
T int | int * T | ( E )
Initialize next to point to first token
Invoke E()
( int )
bool term(TOKEN tok) { return *next++ == tok; }
Notice how this simulates the example parse
Easy to implement by hand
bool E1() { return T(); }
bool E2() { return T() && term(PLUS) && E(); }
bool E() {TOKEN *save = next; return
(next = save, E1())
|| (next = save, E2()); }
bool T1() { return term(INT); }
bool T2() { return term(INT) && term(TIMES) && T(); }
bool T3() { return term(OPEN) && E() && term(CLOSE); }
bool T() { TOKEN *save = next; return
Prof. Aiken CS 143 Lecture 6
52
Prof. Aiken CS 143 Lecture 6
53
(next = save, T1())
|| (next = save, T2())
|| (next = save, T3()); }
Prof. Aiken CS 143 Lecture 6
54
9
When Recursive Descent Does Not Work
Elimination of Left Recursion
Consider a production S S a
Consider the left-recursive grammar
bool S1() { return S() && term(a); }
bool S() { return S1(); }
SS|
S() goes into an infinite loop
S generates all strings starting with a and
followed by a number of
A left-recursive grammar has a non-terminal S
Can rewrite using right-recursion
S S
S + S for some
Recursive descent does not work in such cases
Prof. Aiken CS 143 Lecture 6
S S |
55
Prof. Aiken CS 143 Lecture 6
More Elimination of Left-Recursion
General Left Recursion
In general
56
The grammar
S S 1 | | S n | 1 | | m
All strings derived from S start with one of
1,,m and continue with several instances of
1,,n
Rewrite as
S 1 S | | m S
S 1 S | | n S |
Prof. Aiken CS 143 Lecture 6
SA|
AS
is also left-recursive because
S + S
This left-recursion can also be eliminated
See Dragon Book for general algorithm
Section 4.3
57
Prof. Aiken CS 143 Lecture 6
58
Summary of Recursive Descent
Simple and general parsing strategy
Left-recursion must be eliminated first
but that can be done automatically
Unpopular because of backtracking
Thought to be too inefficient
In practice, backtracking is eliminated by
restricting the grammar
Prof. Aiken CS 143 Lecture 6
59
10