Java Reference
In-Depth Information
Customizing a Stream Tokenizer
The default tokenizing mode can be modified by calling one or other of the following methods:
Method
Description
resetSyntax()
Resets the state of the tokenizer object so no characters have any
special significance. This has the effect that all characters are
regarded as ordinary, and will be read from the stream as single
characters. The value of each character will be stored in the
ttype
field.
ordinaryChar
(int
ch)
Sets the character,
ch
, as an ordinary character. An ordinary
character is a character that has no special significance. It will be
read as a single character whose value will be stored in the
ttype
field. Calling this method will not alter the state of characters other
than the argument value.
ordinaryChars
(int
low,
int
hi)
Causes all characters from
low
to
hi
inclusive to be treated as
ordinary characters. Calling this method will not alter the state of
characters other than those specified by the argument values.
whitespaceChars
(int
low,
int
hi)
Causes all characters from
low
to
hi
inclusive to be treated as
whitespace characters. Unless they appear in a string, whitespace
characters are treated as delimiters between tokens. Calling this
method will not alter the state of characters other than those
specified by the argument values.
wordChars
(int
low,
int
hi)
Specifies that the characters from
low
to
hi
inclusive are word
characters. A word is at least one of these characters. Calling this
method will not alter the state of characters other than those
specified by the argument values.
commentChar(int
ch)
Specifies that
ch
is a character that indicates the start of a
comment. All characters to the end of the line following the
character,
ch
, will be ignored. Calling this method will not alter
the state of characters other than the argument value.
quoteChar(int
ch)
Specifies that matching pairs of the character,
ch
, enclose a string.
Calling this method will not alter the state of characters other than
the argument value.
slashStarComments
(boolean
flag)
If the argument is
false
, this switches off recognizing comments
between /* and */. A
true
argument switches it on again.
slashSlashComments
(boolean
flag)
If the argument is
false
, this switches off recognizing comments
starting will a double slash. A
true
argument switches it on again.
lowerCaseMode
(boolean
flag)
An argument of
true
causes strings to be converted to lower case
before being stored in
sval
. An argument of
false
switches off
lower case mode.
pushback()
Calling this method causes the next call of the
nextToken()
method to return the
ttype
value that was set by the previous
nextToken()
call and to leave
sval
and
nval
unchanged.
