Lexical Analysis - Introduction to Compiler Construction in a Java World - page 33

returnnewTokenInfo(NEW,line);
}
}
}
elseif(ch=='u'){
buffer.append(ch);
nextCh();
if(ch=='l'){
buffer.append(ch);
nextCh();
if(ch=='l'){
buffer.append(ch);
nextCh();
if(!isLetter(ch)&&!isDigit(ch)&&
ch!='_'&&ch!= '$'){
returnnewTokenInfo(NULL,line);
}
}
}
}
while(isLetter(ch)||isDigit(ch)||
ch=='_'||ch=='$'){
buffer.append(ch);
nextCh();
}
returnnewTokenInfo(IDENTIFIER,buffer.toString(),
line);
}
else...
Unfortunately, such state transition diagrams and the corresponding code are too com-
plex. 1 Imagine the code necessary for recognizing all reserved words.
A second way is much more straightforward for hand-written token scanners. We simply
recognize a simple identifier, which may or may not be one of the reserved words. We look
up that identifier in a table of reserved words. If it is there in the table, then we return the
corresponding reserved word. Otherwise, we return the (non-reserved) identifier. The state
transition diagram then looks something like that in Figure 2.3.
FIGURE 2.3 Recognizing words and looking them up in a table to see if they are reserved.
1 Unless the diagrams (and corresponding code) are automatically generated from a specification; we do
this later in the chapter when we visit regular expressions and finite state automata.