Java Reference
In-Depth Information
The method can throw an
IOException
so the call is in a
try
block. The value returned depends on the
token recognized, and from the value, you can determine where to find the token itself. In the fragment
above we compare the value returned with the static constant
TT
_
EOF
that is defined in the StreamTokenizer
class. This value is returned when the end of the stream has been reached. The token that was read from the
stream is itself stored in one of two instance variables of the
StreamTokenizer
object. If the data item is a
number, it is stored in a public data member,
nval
, which is of type
double
. If the data item is a quoted
string or a word, a reference to a
String
object is stored in the public data member,
sval
, which of course
is of type,
String
. The analysis that segments the stream into tokens is fairly simple, and the way in which
an arbitrary stream is broken into tokens is illustrated below.
ignored
String
e3
null
null
comment
ignored
12e3
+
'5'?"
9
"
+
5/
/
w
hat/
n
number
12
String
5
String
9
ignored
null
number
5
ignored
ignored
12.3
-50/
*co
m
m
ent
*//
n
number
12.3
number
-50
As we have said, the
int
value returned by the
nextToken()
method indicates what kind of data item
was read. It can be any of the following constant values defined in the
StreamTokenizer
class:
Token
Value
Description
TT
_
NUMBER
The token is a number that has been stored in the public field,
nval
,
of type
double
, in the tokenizer object.
TT
_
WORD
The token is a word that has been stored in the public field,
sval
, of
type
String
, in the tokenizer object.
TT
_
EOF
The end-of-the stream has been reached.
TT
_
EOL
An end of line character has been read. This is only set if the
eolIsSignificant()
method has been called with the argument,
true
. Otherwise end-of-line characters are treated as whitespace and
ignored.
If a quoted string is read from the stream, the value that is returned by
nextToken()
will be the quote
character used for the string as type
int
- either a single or a double quote. In this case, you retrieve
the reference to the string that was read from the
sval
member of the tokenizer object. The value
indicating what kind of token was read last is also available from a public data member,
ttype
, of the
StreamTokenizer
object, which is of type
int
.
