t.w.x.x.Scanner : class documentation

Part of twisted.words.xish.xpathparser View Source View In Hierarchy

Known subclasses: twisted.words.xish.xpathparser.XPathParserScanner

Yapps scanner.

The Yapps scanner can work in context sensitive or context insensitive modes. The token(i) method is used to retrieve the i-th token. It takes a restrict set that limits the set of tokens it is allowed to return. In context sensitive mode, this restrict set guides the scanner. In context insensitive mode, there is no restriction (the set is always the full set of tokens).

Method __init__ Initialize the scanner.
Method get_token_pos Get the current token position in the input text.
Method get_char_pos Get the current char position in the input text.
Method get_prev_char_pos Get the previous position (one token back) in the input text.
Method get_line_number Get the line number of the current position in the input text.
Method get_column_number Get the column number of the current position in the input text.
Method get_input_scanned Get the portion of the input that has been tokenized.
Method get_input_unscanned Get the portion of the input that has not yet been tokenized.
Method token Get the i'th token in the input.
Method __repr__ Print the last 10 tokens that have been scanned in
Method scan Should scan another token and add it to the list, self.tokens, and add the restriction to self.restrictions
def __init__(self, patterns, ignore, input): (source)
Initialize the scanner.
Parameterspatterns[(terminal, uncompiled regex), ...] or None
ignore[terminal,...]
inputstring

If patterns is None, we assume that the subclass has defined self.patterns : [(terminal, compiled regex), ...]. Note that the patterns parameter expects uncompiled regexes, whereas the self.patterns field expects compiled regexes.

def get_token_pos(self): (source)
Get the current token position in the input text.
def get_char_pos(self): (source)
Get the current char position in the input text.
def get_prev_char_pos(self, i=None): (source)
Get the previous position (one token back) in the input text.
def get_line_number(self): (source)
Get the line number of the current position in the input text.
def get_column_number(self): (source)
Get the column number of the current position in the input text.
def get_input_scanned(self): (source)
Get the portion of the input that has been tokenized.
def get_input_unscanned(self): (source)
Get the portion of the input that has not yet been tokenized.
def token(self, i, restrict=None): (source)
Get the i'th token in the input.

If i is one past the end, then scan for another token.

Parametersitoken index
restrict[token, ...] or None; if restrict is None, then any token is allowed. You may call token(i) more than once. However, the restrict set may never be larger than what was passed in on the first call to token(i).
def __repr__(self): (source)
Print the last 10 tokens that have been scanned in
def scan(self, restrict): (source)
Should scan another token and add it to the list, self.tokens, and add the restriction to self.restrictions
API Documentation for Twisted, generated by pydoctor at 2012-09-01 11:44:56.