Free Trial

Safari Books Online is a digital library providing on-demand subscription access to thousands of learning resources.

  • Create BookmarkCreate Bookmark
  • Create Note or TagCreate Note or Tag
  • PrintPrint

Tokens

Tokens or terminal symbols are symbols that the lexer passes to the parser. Whenever a bison parser needs another token, it calls yylex(), which returns the next token from the input. At the end of input, yylex() returns zero.

Tokens may be either symbols defined by %token or individual characters in single quotes. (See Literal Tokens.) All symbols used as tokens must be defined explicitly in the definition section, for example:

    %token UP DOWN LEFT RIGHT

Tokens can also be declared by %left, %right, or %nonassoc declarations, each of which has exactly the same syntax options as %token has. See Precedence and Associativity Declarations.

Token Numbers

Within the lexer and parser, tokens are identified by small integers. The token number of a literal token is the numeric value in the local character set, usually ASCII, and is the same as the C value of the quoted character.


  

You are currently reading a PREVIEW of this book.

                                                                                                                    

Get instant access to over $1 million worth of books and videos.

  

Start a Free Trial


  
  • Safari Books Online
  • Create BookmarkCreate Bookmark
  • Create Note or TagCreate Note or Tag
  • PrintPrint