javascript：如何在antlr生成的Lexer中进行错误处理？

Question

I am writing a java script grammar parser using antlr. Antlr creates a method for each token in lexer . But it does not contain a catch block. I want to add customize catch block to lexer methods. I have added customized catch block for parser methods , but not able to add it for lexer methods through grammar file.

Here is grammar file -

grammar test;

options {
 language=JavaScript;
 output=AST;
}
@lexer::members {
  function reportError(e) {
    throw e;
  }
}
SALUTATION:'Hello world';  
ENDSYMBOL:'!';
expression : SALUTATION ENDSYMBOL;
catch [e] {
  throw e;
}

And complete demo here - Plunker

Answer 1

TL;DR

Lexer is different than parser -

• When a parser finds an unexpected input, it is in some rule and knows that should have been on the input next
• When a lexer finds an unexpected input, that means that none of the defined lexer rules match the input (that is lexer's definition of "unexpected" - remember that lexer only cares about matching the next "word" on input and doesn't care about what was matched before) - and since the problem is "none of the rules match this", it doesn't make sense to ask "In which lexer rule the problem occurred?"

And because you cannot tell in which lexer rule an unexpected input occurred, there's no point in adding a catch block to them.

---

Long explanation

Adding catch blocks to methods for lexer rules doesn't make much sense. The reason is that the lexer methods are used in a different way than parser methods - this is what happens when the parser queries for the next token:

• org.antlr.runtime.Lexer.nextToken() is called - this method will run the lexer using mTokens() and then collect the result
• mTokens() method in your lexer decides which token rule is appropriate and runs it
  • The decision is done in mTokens() (usually through a DFA - finite automate), not in the lexer rule methods
    • This will throw NoViableAlt exception if no lexer rule matches the current input
• Appropriate lexer rule method consumes the input and sets the appropriate flags for nextToken() to use as the lexer result
  • While this method may throw an exception as well, this should normally never happen since the decision in mTokens() should already choose a matching rule

My understanding is that unless you use syntactic or semantic predicates or custom code in your lexer rules, the lexer rule methods should never actually throw a RecognitionException - the place where these are emitted should be the decision block in mTokens() . So adding a catch block to lexer rule methods makes no sense.

---

mTokens() has no try block around the decision-making part, so no custom catch block is possible unless you modify the code-generation templates. You could try to override org.antlr.runtime.Lexer.nextToken() with your own method with custom exception handling, but there you will only be able to provide one catch block for all lexer rules, not for one specific rule.