Game Development Reference
In-Depth Information
// letters, numbers, and underscores (identifier in
C++)
[a-zA-Z_][a-zA-Z0-9_]*
In any event, once a list of tokens and the regular expressions corresponding to
them is created, this data can then be fed to a program such a flex in order to auto-
matically generate a tokenizer. Once a script is fed into this tokenizer and broken
down into tokens, it's time to move on to the next step.
Syntax Analysis
The job of syntax analysis is to go through the stream of tokens and ensure that
they conform to the rules of the language's grammar. For example, an if state-
ment must have the appropriate number and placement of parentheses, braces, a
test expression, and a statement to execute. As the syntax of the script is verified,
an abstract syntax tree (AST) is generated, which is a tree-based data structure
that defines the layout of the entire program. An AST for a simple math expres-
sion is illustrated in Figure 11.3 .
Figure 11.3 AST for 5 + 6 * 10.
Notice how if the tree in Figure 11.3 were traversed with a post-order traversal
(left child, right child, parent), the result would be 5 6 10 * + , which happens
to correspond how the infix expression 5 + 6 * 10 would be written in
postfix notation. This is not by chance—it turns out that postfix expressions are
Search WWH ::




Custom Search