02 LexicalAnalysis

02 LexicalAnalysis - Lexical Analysis COP 5555 Fall 09...

Info iconThis preview shows pages 1–11. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Lexical Analysis COP 5555 Fall 09 Reading: 2.1-2.2 in Scott In our project, we will implement a scanner as a DFA Skim section on converting NFA to DFA. 2 Recall: Overview of language translation Done in (conceptually) several phases Lexical analysis Performed by a lexer or scanner Converts a sequence of characters into a sequence of tokens Parsing Performed by a parser Recognizes whether tokens have correct phrase structure Converts sequence of tokens into an AST AST = abstract syntax tree The AST is a useful structure for representing the program being translated Static semantic analysis Type checking Code generation may involve several phases, depending on how much optimization is done 3 Syntax and Semantics Syntax describes the structure of a language. Semantics describes the meaning . It is convenient to consider syntax and semantics separately. Syntax is much easier to describe 4 Syntax lexical structure : legal tokens (words) phrase structure : how tokens may be arranged Example: if (balance < 0) { return Account overdrawn; } 5 token kind if keyword ( left paren balance identifier < operator numeric literal ) right paren { left brace return keyword Account overdrawn string literal ; semi } right brace Lexical structure defines how the character sequence is divided into sequence of tokens 6 Phrase structure if (balance < 0) { return Account overdrawn; } According to phrase structure, this is a legal statement because (among other things) the if keyword is followed by an expression enclosed in parentheses , which is followed by a block enclosed in braces , which contains a statement terminated with a ; 7 Formalisms for specifying syntax We use regular expressions for the lexical structure Context-free grammars for the phrase structure (Sometimes people use CFG for both) 8 Regular Expressions Used to specify lexical structure A regular expression is one of ( atom ) a character (an element of an alphabet ) Let = {a,b,c} So far, we have 3 REs a generates the string a b generates the string b c generates the string c ( empty string ) 9 ( concatenation ) two adjacent REs yield any string generated by the first, followed by any string generated by the second....
View Full Document

Page1 / 43

02 LexicalAnalysis - Lexical Analysis COP 5555 Fall 09...

This preview shows document pages 1 - 11. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online