The tokens available from XL.
Different states of the lexer state machine. Basically, these states are the states of reading a floating point number, with a prefix for an integer, and an escape for a non-numeric input.
The stream being lexed.
The buffer used for building the text of the token.
The most recent integer value retrieved.
The most recent floating value retrieved.
The current line.
The current token.
The next character, i.e. the first character of the next token. This is XLT_EOF for end of stream, or XLT_EPSILON if this should be considered invalid (and read before starting the next token).
id initWithStream InputStream s;
Return the int_value as an int. Any loss of bits is not remarked.
Return the matched text.
Skip space and return the next token.
(pointer, int) (contents, length) readBytes int expected_length post length == expected_length;
Skip whitespace, read a quoted string of bytes ("quoting \\like\\ \"this\"") and return it. The length should match the expected length. Anything unexpected results in the return of a NULL pointer.