Is there a default token available in ANTLR in case it doesn't match any of the already defined Tokens?

Question

I have a requirement wherein I have written the Lexer token as :

 IF_LEXER_TOKEN: ('I')('F') (.)* ('E')('N')('D')_('I')('F')
 ANY :(options {greedy=true;}: .)* ;

But if the input is given as :

IF a>b then a=b END_IF
IF c>d then c=d

In this case the expected behavior is that it should use the token IF_LEXER_TOKEN for first line and ANY token for second line, but instead its considering the ANY token for both lines. Kindly help. Note:Due to some constraints I can't create a parser rule for the above scenario.

Answer 1

No, there is no such default token. But you can easily create it:

ANY: .*?;

Best is to make this non-greedy to allow matching other tokens after that input. Btw: defining a complete sequence in the lexer has several drawbacks, eg error reporting cannot give you a good reason if a lexer rule fails. You have to handle all whitespaces explicitly. And you give up principles like that it normally does not matter how many whitespaces (including line breaks) exist between tokens.