tokens

tokens - Preliminaries Language Overview Tokens and Token...

Info iconThis preview shows pages 1–8. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Preliminaries Language Overview Tokens and Token Separators Lexical Tokens of C ++ Len Blanks CSC 1253 Introduction To Computer Science I Using C ++ Louisiana State University September 7, 2010 Len Blanks Lexical Tokens of C ++ Preliminaries Language Overview Tokens and Token Separators Topics 1 Preliminaries 2 Language Overview 3 Tokens and Token Separators Comments Identifiers Reserved Words Operators and Separators Constants Len Blanks Lexical Tokens of C ++ Preliminaries Language Overview Tokens and Token Separators References The material for this lecture is heavily based on content from The C ++ Programming Language by Bjarne Stroustrup and material found on http://msdn.microsoft.com/en-us/library/3yx2xe3h(VS.80).aspx . Len Blanks Lexical Tokens of C ++ Preliminaries Language Overview Tokens and Token Separators A C ++ program is composed of one or more preprocessed source files or translation units, each of which contains some part of the entire C ++ program — typically some number of external functions. One external function must be named main ; by convention this will be the first function executed. Len Blanks Lexical Tokens of C ++ Preliminaries Language Overview Tokens and Token Separators Comments Identifiers Reserved Words Operators and Separators Constants Tokens and Token Separators A lexical token of a programming language is an atomic symbol of the language—the smallest element of a C ++ program that is meaningful to the compiler. The characters making up a C ++ program are collected into tokens in the lexical analysis phase of the compilation process. There are five classes of tokens: operators, separators, identifiers, reserved words and literals(constants). It might be necessary at times to explicitly separate one token in the program from another; for that purpose the language defines token separators which include space characters, horizontal and vertical tabs, form feeds, newlines and comments. These, except for comments, are known collectively as “white space”. Len Blanks Lexical Tokens of C ++ Preliminaries Language Overview Tokens and Token Separators Comments Identifiers Reserved Words Operators and Separators Constants Tokens and Token Separators The scanner always forms the longest token possible as it collects tokens in a left-to-right order, even if the results do not result in a valid C ++ program. Consider this code fragment: a = i+++j; The programmer who wrote the code might have intended either of these two statements: a = i + (++j); a = (i++) + j; Because the scanner creates the longest token possible from the input stream, it chooses the second interpretation, making the tokens i , ++ , + and j . Len Blanks Lexical Tokens of C ++ Preliminaries Language Overview Tokens and Token Separators Comments Identifiers Reserved Words Operators and Separators Constants C ++ Comments A comment is text that the compiler ignores but is useful to human beings in annotating code for future reference. C ++ has two flavours of comments; the first form is inherited from C; the...
View Full Document

This note was uploaded on 10/02/2011 for the course CSC 1253 taught by Professor Staff during the Fall '08 term at LSU.

Page1 / 25

tokens - Preliminaries Language Overview Tokens and Token...

This preview shows document pages 1 - 8. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online