lex - lexLexical Analysis 2 With the lex(1) software tool...

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon
51 lex –Lexical Analysis 2 With the lex(1) software tool you can solve problems from text processing, code enciphering, compiler writing, and other areas. In text processing, you might check the spelling of words for errors; in code enciphering, you might translate certain patterns of characters into others; and in compiler writing, you might determine what the tokens (smallest meaningful sequences of characters) are in the program to be compiled. The task common to all these problems is lexical analysis: recognizing different strings of characters that satisfy certain characteristics. Hence the name lex . You don’t have to use lex to handle problems of this kind. You could write programs in a standard language like C to handle them, too. In fact, what lex does is produce such C programs. ( lex is therefore called a program generator.) What lex offers you is typically a faster, easier way to create programs that perform these tasks. Its weakness is that it often produces C programs that are longer than necessary for the task at hand and that execute more slowly than they otherwise might. In many applications this is a minor consideration, and the advantages of using lex considerably outweigh it.
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
52 Programming Utilities Guide November 1995 2 lex can also be used to collect statistical data on features of an input text, such as character count, word length, number of occurrences of a word, and so forth. In the remaining sections of this chapter, you will see the following: Generating a lexical analyzer program Writing lex source Translating lex source Using lex with yacc Internationalization For information about using lex to develop applications in languages other than English, see lex (1). Generating a Lexical Analyzer Program lex generates a C-language scanner from a source specification that you write. This specification contains a list of rules indicating sequences of characters — expressions — to be searched for in an input text, and the actions to take when an expression is found. To see how to write a lex specification see the section “Writing lex Source” on page 54. The C source code for the lexical analyzer is generated when you enter where lex.l is the file containing your lex specification. (The name lex.l is conventionally the favorite, but you can use whatever name you want. Keep in mind, though, that the .l suffix is a convention recognized by other system tools, make in particular.) The source code is written to an output file called lex.yy.c by default. That file contains the definition of a function called yylex() that returns 1 whenever an expression you have specified is found in the input text, 0 when end of file is encountered. Each call to yylex() parses one token (assuming a return); when yylex() is called again, it picks up where it left off.
Background image of page 2
Image of page 3
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 12/25/2010 for the course ALL 0204 taught by Professor 79979 during the Spring '10 term at National Chiao Tung University.

Page1 / 30

lex - lexLexical Analysis 2 With the lex(1) software tool...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online