Lecture 11 Notes

243 specifying scanners in sllgen 245 the sllgen

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Generally the scanner is a subroutine that, when called, produces the “next” token of the input. Could write scanner from scratch, but that’s tedious and error-prone. Better idea: write down the lexical specification in a specialized language. Such a specialized language is called a domain-specific language, or DSL. 12 2.4.2 Specification via regular expressions Most common language is a language of regular expressions. Regular expressions are widely used for pattern matching (eg in Perl, grep, etc.). ¦ ¦ ¦ character R R R R R character ¡ ¡¦ ¢ :: R What do these mean? A character c matches the string consisting of the character c. c matches any 1-character string other than c. ¢ RS matches any string that consists of a string matching R followed by a string matching S. (This is called concatenation). R S matches any string that either matches R or matches S. (This is sometimes written R S). ¦ R matches any number n of copies (n the Kleene closure of R. 0) of strings that match R. This is called ¡ Examples: £  £ , , £ £ ¡ cd) matches the empty string, , .... , , , , or , .... ,  , , £ , , £  , . , £  £ £ ¡  (ab ... ,    (ab) matches the empty string, £ £ £  £ £ £  £   £  £ £  (ab cd)(ab cd e f ) matches . £ £ £  £  £  and  £ ab cd matches the strings .  ab matches only the string , Regular expressions for our example: (space newline) (space newline) ¡ ( newline) ¡ whitespace ¢ 13 ' ¥% ¡ Rule for scanners: always take the longest match. This way identifier, not 3. letter (letter digit) ¡ ¡ identifier ¡ comment comes out as one  ¢ % ¤ ¥ © ¨ ¡ §  ¡  £ #  ¥  ¥  ¦¦ ¥¥¦ ¢ £'  ¤ © ¢  ¡ ¢ % ¤ 14 ¦ §¥ ¦ ¥¦ ©   © ¦£ ¦ £  ©  ¥  #¥#"§  ¢   © ¦ ¤ ¨# ¢ ¥ ¡  ¥ ¥ ¥©    ¡¡  ¡  ¡ ¥§©¡  ¥ ¡  ¢¢©£¢ ¢ ¢ ¡ © ¡ ¥ ¡¢   ¦¢©  £ £ ¡ ¤ £ § ¦§ ¦  ¡ ¤ £ § ¥ ¢  # ¥© ¡  ¤¨£ §   ¡  ¢ ¢ ¦© ¢¡ £¡ ¡  ...
View Full Document

This document was uploaded on 03/17/2014 for the course CSG 111 at Northeastern.

Ask a homework question - tutors are online