Class StreamTokenizerCAR

java.lang.Object
  |
  +--StreamTokenizerCAR

All Implemented Interfaces:

CarFace, LexerFace

public class StreamTokenizerCAR

extends java.lang.Object

implements LexerFace

The StreamTokenizerCAR class is based on Java's StreamTokenizer class. It takes an input stream and parses it into "tokens", allowing the tokens to be read one at a time. The parsing process is controlled by a table and a number of flags that can be set to various states. The stream tokenizer can recognize identifiers, numbers, quoted strings, and various comment styles.

Each byte read from the input stream is regarded as a character in the range '\u0000' through '\u00FF'. The character value is used to look up five possible attributes of the character: white space, alphabetic, numeric, string quote, and comment character. Each character can have zero or more of these attributes.

In addition, an instance has four flags. These flags indicate:

• Whether line terminators are to be returned as tokens or treated as white space that merely separates tokens.
• Whether C-style comments are to be recognized and skipped.
• Whether C++-style comments are to be recognized and skipped.
• Whether the characters of identifiers are converted to lowercase.

A typical application first constructs an instance of this class, sets up the syntax tables, and then repeatedly loops calling the nextToken method in each iteration of the loop until it returns the value TT_EOF.

Since:

JDK1.0

See Also:

java.io.StreamTokenizer#nextToken(), java.io.StreamTokenizer#TT_EOF

Field Summary

double	nval
(package private) boolean	pbufIFirstTime
java.lang.String	sval
static int	TT_EOF
static int	TT_EOL
static int	TT_NOTHING
static int	TT_WORD
static int	ttype

Fields inherited from interface LexerFace

ACTIVETAGS, AMPERSAND, any, ANY, COLON, COMMA, COMMENT, DIV, ELLIPSIS, ENDOFSOURCE, ENDTAG, EQUALS, GT, IDENTIFIER, LBRACE, LBRACKET, LPARENT, LT, MATCHERMESSAGE, MINUS, NOT, NOTE, PERIOD, PLUS, QUESTION, RBRACE, RBRACKET, RPARENT, SEMICOLON, STAR, STARTTAG, TILDE, TOKENMIN, tokenNames

Fields inherited from interface CarFace

BOOLEAN_LITERAL, COLLECTION, DOUBLE_LITERAL, EXTERNAL, grammarNames, INTERNAL, KEYPHRASE, LITERALcapacityIncrement, LITERALinitialCapacity, MARKUP, SERIAL, SESSION, STRING_LITERAL, STRINGVALUE_LITERAL

Constructor Summary

StreamTokenizerCAR(java.io.Reader r, boolean traceB, int pgrammar)

Method Summary

void	commentChar(int ch)
void	eolIsSignificant(boolean flag)
int	lineno()
void	lowerCaseMode(boolean fl)
int	nextToken()
void	ordinaryChar(int ch)
void	ordinaryChars(int low, int hi)
void	parseNumbers()
void	pushBack()
void	quoteChar(int ch)
void	resetSyntax()
void	slashSlashComments(boolean flag)
void	slashStarComments(boolean flag)
java.lang.String	toString()
void	whitespaceChars(int low, int hi)
void	wordChars(int low, int hi)

Methods inherited from class java.lang.Object

, clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

TT_NOTHING

public static final int TT_NOTHING

TT_EOF

public static final int TT_EOF

TT_EOL

public static final int TT_EOL

TT_WORD

public static final int TT_WORD

ttype

public static int ttype

sval

public java.lang.String sval

nval

public double nval

pbufIFirstTime

boolean pbufIFirstTime

Constructor Detail

StreamTokenizerCAR

public StreamTokenizerCAR(java.io.Reader r,
                          boolean traceB,
                          int pgrammar)

Method Detail

resetSyntax

public void resetSyntax()

wordChars

public void wordChars(int low,
                      int hi)

whitespaceChars

public void whitespaceChars(int low,
                            int hi)

ordinaryChars

public void ordinaryChars(int low,
                          int hi)

ordinaryChar

public void ordinaryChar(int ch)

commentChar

public void commentChar(int ch)

quoteChar

public void quoteChar(int ch)

parseNumbers

public void parseNumbers()

eolIsSignificant

public void eolIsSignificant(boolean flag)

slashStarComments

public void slashStarComments(boolean flag)

slashSlashComments

public void slashSlashComments(boolean flag)

lowerCaseMode

public void lowerCaseMode(boolean fl)

nextToken

public int nextToken()
              throws java.io.IOException

pushBack

public void pushBack()

lineno

public int lineno()

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object