Class Tokenizer

    • Field Detail

      • redir

        private final java.util.regex.Pattern redir
      • inArray

        protected boolean inArray
      • word

        protected int word
      • pushed

        protected Token pushed
      • last

        protected Token last
    • Constructor Detail

      • Tokenizer

        public Tokenizer​(java.lang.CharSequence text)
    • Method Detail

      • next

        public Token next()
      • token

        private Token token​(int start)
      • push

        public void push​(Token token)
      • readHereDoc

        public Token readHereDoc​(boolean ignoreLeadingTabs)