hlt.language.io

Class LAStreamTokenizer

java.lang.Object

hlt.language.io.AbstractStreamTokenizer

hlt.language.io.LAStreamTokenizer

public class LAStreamTokenizer
extends AbstractStreamTokenizer

This provides the exact same interface as that of the standard Java Core API's java.io.StreamTokenizer but with a simpler (and IMHO sounder) implementation, as well as a more complete and more general set of methods (see AbstractStreamTokenizer). In addition, io.LAStreamTokenizer provides a method for arbitrary lookaheads:

 public final String lookahead (int n) throws IOException
 

which returns the string of n characters in the input stream including and starting with the current character.

Version:

Last modified on Wed Jun 20 13:57:23 2012 by hak

Author:

Hassan Aït-Kaci

See Also:

AbstractStreamTokenizer, StreamTokenizer

Copyright:

© by the author

Field Summary

Fields inherited from class hlt.language.io.AbstractStreamTokenizer

COMMENT, isInteger, NUMERIC, nval, ORDINARY, QUOTE, SPECIAL, sval, TT_EOF, TT_EOL, TT_NOTHING, TT_NUMBER, TT_SPECIAL, TT_WORD, ttype, WHITESPACE, WORD

Constructor Summary

Constructors

Constructor and Description
LAStreamTokenizer(java.io.Reader rd) Creates a StreamTokenizer for the specified Reader and sets the default syntax.

Method Summary

Methods

Modifier and Type	Method and Description
void	commentBrackets(java.lang.String left, java.lang.String right) Enables bracketed comments with the specified start and end strings.
void	commentChar(int c) Makes the specified character a comment start character exclusively and enable skipping comment lines starting with this character.
void	commentLine(java.lang.String s) Enables line comments with the specified start string.
void	disableComments()
void	enableComments()
java.lang.String	lookahead(int n) Returns the string of n characters to be processed next, including and starting with the current character.
int	nextToken() Reads a token and returns its type, which is also the value of ttype.
void	setEscapeChar(int c) Sets the default escape character.
void	setTypePrecedence(int first, int second, int third, int forth) Changes the precedence ordering among the character types to the order of the specified parameters.
void	slashSlashComments(boolean flag) C++-style comments will be skipped iff the argument is true.
void	slashStarComments(boolean flag) C-style comments will be skipped iff the argument is true.

Methods inherited from class hlt.language.io.AbstractStreamTokenizer

currentChar, eolIsSignificant, getFile, getLineNumber, ignoreNonDecimals, isNumericChar, isOrdinaryChar, isQuoteChar, isWhitespaceChar, isWordChar, lineno, location, ordinaryChar, ordinaryChars, ordinaryChars, parseDigitsOnly, parseNonDecimals, parseNonDecimals, parseNumbers, parseNumbers, parsingDigitsOnly, peek, peekBack, pushBack, quoteChar, quoteChar, quotePair, quotePair, resetSyntax, setDefaultSyntax, setOrdinaryChar, setOrdinaryChars, setQuoteChar, setQuoteChar, setQuotePair, setType, setType, setType, setWhitespaceChar, setWhitespaceChars, setWordChar, setWordChars, skipChar, skipChar, spaceIsSignificant, tokenEnd, tokenStart, toString, unsetType, unsetType, unsetType, whitespaceChar, whitespaceChars, whitespaceChars, wordChar, wordChars, wordChars

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

LAStreamTokenizer

public LAStreamTokenizer(java.io.Reader rd)

Creates a StreamTokenizer for the specified Reader and sets the default syntax.

Method Detail

enableComments

public final void enableComments()

Specified by:

enableComments in class AbstractStreamTokenizer

disableComments

public final void disableComments()

Specified by:

disableComments in class AbstractStreamTokenizer

commentChar

public final void commentChar(int c)

Makes the specified character a comment start character exclusively and enable skipping comment lines starting with this character.

Specified by:

commentChar in class AbstractStreamTokenizer

slashSlashComments

public final void slashSlashComments(boolean flag)

C++-style comments will be skipped iff the argument is true.

Specified by:

slashSlashComments in class AbstractStreamTokenizer

slashStarComments

public final void slashStarComments(boolean flag)

C-style comments will be skipped iff the argument is true.

Specified by:

slashStarComments in class AbstractStreamTokenizer

nextToken

public final int nextToken()
                    throws java.io.IOException

Reads a token and returns its type, which is also the value of ttype. The possible token types are:

TT_NOTHING

if no token has been read yet;

TT_EOF

if the end of file has been read;

TT_EOL

if the end of line is significant and has been read;

TT_WORD

if a word has been read;

TT_NUMBER

if number parsing is effective and a number has been read;

a quote character

if a quoted word has been read;

a white space

if white spaces are significant and one has been read;

an ordinary character

otherwise.

If the type is TT_WORD or a quote character, the value of the word read is in sval. If the type is TT_NUMBER, the value of the number is in nval. If the number read is an integer, the flag isInteger is set to true.

Specified by:

nextToken in class AbstractStreamTokenizer

Throws:

java.io.IOException

setEscapeChar

public final void setEscapeChar(int c)

Sets the default escape character.

commentBrackets

public final void commentBrackets(java.lang.String left,
                   java.lang.String right)

Enables bracketed comments with the specified start and end strings.

commentLine

public final void commentLine(java.lang.String s)

Enables line comments with the specified start string.

setTypePrecedence

public void setTypePrecedence(int first,
                     int second,
                     int third,
                     int forth)
                       throws java.lang.Exception

Changes the precedence ordering among the character types to the order of the specified parameters. For example, the default ordering is set by:

setTypePrecedence(WORD,WHITESPACE,QUOTE,ORDINARY);

and this means that the WORD type is considered in priority, then WHITESPACE, etc...

The NUMERIC type is not among those that can be ordered because it is relevant only when numbers are parsed. If they are, then the full Java number format is recognized independently of the character syntax setting. This means that NUMERIC has always highest precedence if number parsing is effective, and ignored otherwise.

Throws:

java.lang.Exception

lookahead

public final java.lang.String lookahead(int n)
                                 throws java.io.IOException

Returns the string of n characters to be processed next, including and starting with the current character.

Throws:

java.io.IOException