Class AssignTokenTypesWalker

  • All Implemented Interfaces:
    AssignTokenTypesWalkerTokenTypes
    Direct Known Subclasses:
    AssignTokenTypesBehavior

    public class AssignTokenTypesWalker
    extends antlr.TreeParser
    implements AssignTokenTypesWalkerTokenTypes
    [Warning: TJP says that this is probably out of date as of 11/19/2005, but since it's probably still useful, I'll leave in. Don't have energy to update at the moment.] Compute the token types for all literals and rules etc.. There are a few different cases to consider for grammar types and a few situations within. CASE 1 : pure parser grammar a) Any reference to a token gets a token type. b) The tokens section may alias a token name to a string or char CASE 2 : pure lexer grammar a) Import token vocabulary if available. Set token types for any new tokens to values above last imported token type b) token rule definitions get token types if not already defined c) literals do NOT get token types CASE 3 : merged parser / lexer grammar a) Any char or string literal gets a token type in a parser rule b) Any reference to a token gets a token type if not referencing a fragment lexer rule c) The tokens section may alias a token name to a string or char which must add a rule to the lexer d) token rule definitions get token types if not already defined e) token rule definitions may also alias a token name to a literal. E.g., Rule 'FOR : "for";' will alias FOR to "for" in the sense that references to either in the parser grammar will yield the token type What this pass does: 0. Collects basic info about the grammar like grammar name and type; Oh, I have go get the options in case they affect the token types. E.g., tokenVocab option. Imports any token vocab name/type pairs into a local hashtable. 1. Finds a list of all literals and token names. 2. Finds a list of all token name rule definitions; no token rules implies pure parser. 3. Finds a list of all simple token rule defs of form " : ;" and aliases them. 4. Walks token names table and assign types to any unassigned 5. Walks aliases and assign types to referenced literals 6. Walks literals, assigning types if untyped 4. Informs the Grammar object of the type definitions such as: g.defineToken(, ttype); g.defineToken(, ttype); g.defineToken(, ttype); where some of the ttype values will be the same for aliases tokens.
    • Field Detail

      • grammar

        protected Grammar grammar
      • currentRuleName

        protected java.lang.String currentRuleName
      • stringAlias

        protected static GrammarAST stringAlias
      • charAlias

        protected static GrammarAST charAlias
      • stringAlias2

        protected static GrammarAST stringAlias2
      • charAlias2

        protected static GrammarAST charAlias2
      • _tokenNames

        public static final java.lang.String[] _tokenNames
    • Constructor Detail

      • AssignTokenTypesWalker

        public AssignTokenTypesWalker()
    • Method Detail

      • reportError

        public void reportError​(antlr.RecognitionException ex)
        Overrides:
        reportError in class antlr.TreeParser
      • initASTPatterns

        protected void initASTPatterns()
      • trackString

        protected void trackString​(GrammarAST t)
      • trackToken

        protected void trackToken​(GrammarAST t)
      • defineTokens

        public void defineTokens​(Grammar root)
      • defineStringLiteralsFromDelegates

        protected void defineStringLiteralsFromDelegates()
      • assignStringTypes

        protected void assignStringTypes​(Grammar root)
      • aliasTokenIDsAndLiterals

        protected void aliasTokenIDsAndLiterals​(Grammar root)
      • assignTokenIDTypes

        protected void assignTokenIDTypes​(Grammar root)
      • defineTokenNamesAndLiteralsInGrammar

        protected void defineTokenNamesAndLiteralsInGrammar​(Grammar root)
      • init

        protected void init​(Grammar root)
      • grammar

        public final void grammar​(antlr.collections.AST _t,
                                  Grammar g)
                           throws antlr.RecognitionException
        Throws:
        antlr.RecognitionException
      • grammarSpec

        public final void grammarSpec​(antlr.collections.AST _t)
                               throws antlr.RecognitionException
        Throws:
        antlr.RecognitionException
      • optionsSpec

        public final java.util.Map optionsSpec​(antlr.collections.AST _t)
                                        throws antlr.RecognitionException
        Throws:
        antlr.RecognitionException
      • delegateGrammars

        public final void delegateGrammars​(antlr.collections.AST _t)
                                    throws antlr.RecognitionException
        Throws:
        antlr.RecognitionException
      • tokensSpec

        public final void tokensSpec​(antlr.collections.AST _t)
                              throws antlr.RecognitionException
        Throws:
        antlr.RecognitionException
      • attrScope

        public final void attrScope​(antlr.collections.AST _t)
                             throws antlr.RecognitionException
        Throws:
        antlr.RecognitionException
      • rules

        public final void rules​(antlr.collections.AST _t)
                         throws antlr.RecognitionException
        Throws:
        antlr.RecognitionException
      • option

        public final void option​(antlr.collections.AST _t,
                                 java.util.Map opts)
                          throws antlr.RecognitionException
        Throws:
        antlr.RecognitionException
      • optionValue

        public final java.lang.Object optionValue​(antlr.collections.AST _t)
                                           throws antlr.RecognitionException
        Throws:
        antlr.RecognitionException
      • charSet

        public final void charSet​(antlr.collections.AST _t)
                           throws antlr.RecognitionException
        Throws:
        antlr.RecognitionException
      • charSetElement

        public final void charSetElement​(antlr.collections.AST _t)
                                  throws antlr.RecognitionException
        Throws:
        antlr.RecognitionException
      • tokenSpec

        public final void tokenSpec​(antlr.collections.AST _t)
                             throws antlr.RecognitionException
        Throws:
        antlr.RecognitionException
      • rule

        public final void rule​(antlr.collections.AST _t)
                        throws antlr.RecognitionException
        Throws:
        antlr.RecognitionException
      • modifier

        public final void modifier​(antlr.collections.AST _t)
                            throws antlr.RecognitionException
        Throws:
        antlr.RecognitionException
      • ruleScopeSpec

        public final void ruleScopeSpec​(antlr.collections.AST _t)
                                 throws antlr.RecognitionException
        Throws:
        antlr.RecognitionException
      • block

        public final void block​(antlr.collections.AST _t)
                         throws antlr.RecognitionException
        Throws:
        antlr.RecognitionException
      • exceptionGroup

        public final void exceptionGroup​(antlr.collections.AST _t)
                                  throws antlr.RecognitionException
        Throws:
        antlr.RecognitionException
      • alternative

        public final void alternative​(antlr.collections.AST _t)
                               throws antlr.RecognitionException
        Throws:
        antlr.RecognitionException
      • rewrite

        public final void rewrite​(antlr.collections.AST _t)
                           throws antlr.RecognitionException
        Throws:
        antlr.RecognitionException
      • element

        public final void element​(antlr.collections.AST _t)
                           throws antlr.RecognitionException
        Throws:
        antlr.RecognitionException
      • exceptionHandler

        public final void exceptionHandler​(antlr.collections.AST _t)
                                    throws antlr.RecognitionException
        Throws:
        antlr.RecognitionException
      • finallyClause

        public final void finallyClause​(antlr.collections.AST _t)
                                 throws antlr.RecognitionException
        Throws:
        antlr.RecognitionException
      • atom

        public final void atom​(antlr.collections.AST _t)
                        throws antlr.RecognitionException
        Throws:
        antlr.RecognitionException
      • ebnf

        public final void ebnf​(antlr.collections.AST _t)
                        throws antlr.RecognitionException
        Throws:
        antlr.RecognitionException
      • tree

        public final void tree​(antlr.collections.AST _t)
                        throws antlr.RecognitionException
        Throws:
        antlr.RecognitionException
      • ast_suffix

        public final void ast_suffix​(antlr.collections.AST _t)
                              throws antlr.RecognitionException
        Throws:
        antlr.RecognitionException