Uses of Class
sjm.parse.tokens.Token

Packages that use Token

sjm.examples.reserved
sjm.examples.sling
sjm.examples.tokens
sjm.parse.tokens

Uses of Token in sjm.examples.reserved

Methods in sjm.examples.reserved that return Token

Token

WordOrReservedState.nextToken(java.io.PushbackReader r, int c, Tokenizer t) Return a reserved token or a word token from a reader.

Uses of Token in sjm.examples.sling

Fields in sjm.examples.sling declared as Token

protected Token

ReservedLiteral.literal the literal to match

Uses of Token in sjm.examples.tokens

Methods in sjm.examples.tokens that return Token

protected Token

ScientificNumberState.value(java.io.PushbackReader r, Tokenizer t)

Uses of Token in sjm.parse.tokens

Fields in sjm.parse.tokens declared as Token

protected Token	Symbol.symbol the literal to match
protected Token	Literal.literal the literal to match
protected Token[]	TokenString.tokens the tokens in this tokenString
static Token	Token.EOF A constant indicating that there are no more tokens

Methods in sjm.parse.tokens that return Token

Token	Tokenizer.nextToken() Return the next token.
abstract Token	TokenizerState.nextToken(java.io.PushbackReader r, int c, Tokenizer t) Return a token that represents a logical piece of a reader.
Token	QuoteState.nextToken(java.io.PushbackReader r, int cin, Tokenizer t) Return a quoted string token from a reader.
Token	NumberState.nextToken(java.io.PushbackReader r, int cin, Tokenizer t) Return a number token from a reader.
protected Token	NumberState.value(java.io.PushbackReader r, Tokenizer t)
Token	TokenString.tokenAt(int i) Returns the token at the specified index.
Token	SlashStarState.nextToken(java.io.PushbackReader r, int theStar, Tokenizer t) Ignore everything up to a closing star and slash, and then return the tokenizer's next token.
Token	SlashState.nextToken(java.io.PushbackReader r, int theSlash, Tokenizer t) Either delegate to a comment-handling state, or return a token with just a slash in it.
Token	SlashSlashState.nextToken(java.io.PushbackReader r, int theSlash, Tokenizer t) Ignore everything up to an end-of-line and return the tokenizer's next token.
Token	SymbolState.nextToken(java.io.PushbackReader r, int first, Tokenizer t) Return a symbol token from a reader.
Token	WordState.nextToken(java.io.PushbackReader r, int c, Tokenizer t) Return a word token from a reader.
Token	WhitespaceState.nextToken(java.io.PushbackReader r, int aWhitespaceChar, Tokenizer t) Ignore whitespace (such as blanks and tabs), and return the tokenizer's next token.

Constructors in sjm.parse.tokens with parameters of type Token

TokenString.TokenString(Token[] tokens)
Constructs a tokenString from the supplied tokens.